Jul 21 18:00:24.795763 ip-10-0-139-4 systemd[1]: Starting Kubernetes Kubelet... Jul 21 18:00:25.263298 ip-10-0-139-4 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 18:00:25.263298 ip-10-0-139-4 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 18:00:25.263298 ip-10-0-139-4 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 18:00:25.263298 ip-10-0-139-4 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 18:00:25.263298 ip-10-0-139-4 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 18:00:25.264594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.264459 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 18:00:25.271275 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271259 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 18:00:25.271275 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271275 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271278 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271282 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271285 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271289 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271292 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271295 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271298 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271301 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271304 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271306 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271309 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271313 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271317 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271321 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271324 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271327 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271331 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271334 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 18:00:25.271348 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271337 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271340 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271343 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271345 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271349 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271353 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271357 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271360 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271364 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271367 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271369 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271372 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271374 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271377 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271379 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271382 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271384 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271387 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271390 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271393 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 18:00:25.271815 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271395 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271398 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271400 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271402 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271405 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271407 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271409 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271412 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271414 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271416 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271419 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271421 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271423 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271426 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271429 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271432 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271434 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271437 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271440 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271453 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 18:00:25.272295 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271456 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271458 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271461 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271463 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271466 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271468 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271472 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271475 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271477 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271479 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271482 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271484 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271487 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271489 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271492 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271494 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271496 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271499 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271501 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271504 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 18:00:25.272828 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271506 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271508 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271511 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271514 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271516 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271518 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271522 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271525 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271528 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271530 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271533 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271536 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271538 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271540 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271543 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271546 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271548 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271550 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.271553 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 18:00:25.273318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272257 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272262 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272265 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272268 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272271 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272274 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272279 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272283 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272286 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272289 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272292 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272295 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272298 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272301 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272304 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272307 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272309 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272312 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272314 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272318 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 18:00:25.273817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272320 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272323 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272325 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272328 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272330 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272333 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272335 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272338 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272340 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272343 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272346 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272348 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272350 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272353 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272356 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272358 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272360 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272363 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272366 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 18:00:25.274353 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272368 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272371 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272375 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272378 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272380 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272383 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272385 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272388 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272390 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272392 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272395 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272397 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272400 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272402 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272405 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272407 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272410 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272412 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272415 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 18:00:25.274864 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272418 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272420 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272423 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272425 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272428 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272430 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272433 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272435 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272438 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272440 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272459 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272462 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272465 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272469 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272472 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272475 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272478 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272481 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272484 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272487 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 18:00:25.275332 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272489 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272492 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272494 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272497 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272500 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272502 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272505 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272507 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272510 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272514 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272516 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272519 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272522 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272524 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272527 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272529 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272532 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272534 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272536 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272539 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 18:00:25.275833 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272541 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272940 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272945 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272949 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272952 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272959 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272962 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272965 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272968 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272971 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272973 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272976 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272980 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272983 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272986 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272989 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272991 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272995 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.272998 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273000 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 18:00:25.276319 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273003 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273006 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273008 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273011 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273013 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273016 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273018 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273021 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273023 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273026 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273028 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273031 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273033 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273036 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273038 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273041 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273043 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273046 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273049 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273051 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 18:00:25.276826 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273053 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273056 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273059 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273061 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273064 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273066 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273069 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273074 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273085 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273109 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273112 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273115 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273117 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273120 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273122 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273125 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273128 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273131 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273135 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273137 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 18:00:25.277312 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273140 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273142 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273144 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273147 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273149 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273153 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273158 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273162 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273165 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273168 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273171 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273178 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273181 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273183 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273185 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273188 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273190 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273193 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273195 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 18:00:25.277913 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273199 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273201 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273204 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273207 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273209 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273212 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273214 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273217 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273219 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273221 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273224 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273227 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273230 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273232 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273235 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273237 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273240 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273242 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273245 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 18:00:25.278378 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273248 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273250 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273315 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273322 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273328 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273332 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273349 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273352 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273357 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273361 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273364 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273366 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273370 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273374 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273377 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273379 2560 flags.go:64] FLAG: --cgroup-root="" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273382 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273385 2560 flags.go:64] FLAG: --client-ca-file="" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273388 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273390 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273395 2560 flags.go:64] FLAG: --cluster-domain="" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273398 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273401 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273404 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 18:00:25.278852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273407 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273410 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273413 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273416 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273419 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273422 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273425 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273428 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273431 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273434 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273438 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273455 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273458 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273461 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273464 2560 flags.go:64] FLAG: --enable-server="true" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273467 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273471 2560 flags.go:64] FLAG: --event-burst="100" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273474 2560 flags.go:64] FLAG: --event-qps="50" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273477 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273480 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273483 2560 flags.go:64] FLAG: --eviction-hard="" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273487 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273489 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273494 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273497 2560 flags.go:64] FLAG: --eviction-soft="" Jul 21 18:00:25.279409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273500 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273503 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273506 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273509 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273511 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273514 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273517 2560 flags.go:64] FLAG: --feature-gates="" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273521 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273524 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273527 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273530 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273533 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273536 2560 flags.go:64] FLAG: --help="false" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273539 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273542 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273545 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273547 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273550 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273554 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273556 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273559 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273561 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273564 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 18:00:25.280018 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273567 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273570 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273573 2560 flags.go:64] FLAG: --kube-reserved="" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273576 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273578 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273582 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273584 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273589 2560 flags.go:64] FLAG: --lock-file="" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273591 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273594 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273597 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273602 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273605 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273608 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273611 2560 flags.go:64] FLAG: --logging-format="text" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273613 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273616 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273619 2560 flags.go:64] FLAG: --manifest-url="" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273622 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273626 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273629 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273633 2560 flags.go:64] FLAG: --max-pods="110" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273636 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273640 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273642 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 18:00:25.280696 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273645 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273648 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273651 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273653 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273660 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273663 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273666 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273668 2560 flags.go:64] FLAG: --pod-cidr="" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273672 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273677 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273680 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273683 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273686 2560 flags.go:64] FLAG: --port="10250" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273689 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273693 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bcc4d5c04314bcd1" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273696 2560 flags.go:64] FLAG: --qos-reserved="" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273699 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273702 2560 flags.go:64] FLAG: --register-node="true" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273705 2560 flags.go:64] FLAG: --register-with-taints="" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273708 2560 flags.go:64] FLAG: --registry-burst="10" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273710 2560 flags.go:64] FLAG: --registry-qps="5" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273713 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273716 2560 flags.go:64] FLAG: --reserved-memory="" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273719 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273722 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 18:00:25.281317 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273725 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273728 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273730 2560 flags.go:64] FLAG: --runonce="false" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273733 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273736 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273738 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273741 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273744 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273746 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273749 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273752 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273754 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273757 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273760 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273762 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273765 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273769 2560 flags.go:64] FLAG: --system-cgroups="" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273771 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273776 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273779 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273784 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273790 2560 flags.go:64] FLAG: --tls-min-version="" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273793 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273795 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273798 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 18:00:25.281930 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273801 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273803 2560 flags.go:64] FLAG: --v="2" Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273807 2560 flags.go:64] FLAG: --version="false" Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273811 2560 flags.go:64] FLAG: --vmodule="" Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273815 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.273818 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273955 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273960 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273963 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273967 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273971 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273974 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273977 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273979 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273982 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273984 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273987 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273989 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273992 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273994 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273997 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 18:00:25.282541 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.273999 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274002 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274004 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274014 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274018 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274021 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274026 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274030 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274033 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274036 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274039 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274041 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274043 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274046 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274048 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274051 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274053 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274055 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274058 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 18:00:25.283057 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274061 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274063 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274066 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274068 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274071 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274073 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274075 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274078 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274080 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274083 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274085 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274088 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274090 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274093 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274095 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274098 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274100 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274110 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274112 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274116 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 18:00:25.283547 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274120 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274123 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274125 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274128 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274130 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274133 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274135 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274137 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274140 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274142 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274144 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274147 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274149 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274152 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274154 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274157 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274159 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274161 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274164 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274167 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 18:00:25.284068 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274169 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274171 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274174 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274176 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274179 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274181 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274183 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274186 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274188 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274192 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274197 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274201 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274205 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274207 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274210 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274212 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274215 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274217 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274220 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 18:00:25.284563 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274222 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274224 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274227 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274230 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274232 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.274235 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.275211 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.282346 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.282365 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.282385 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.282391 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282477 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282483 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282486 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282489 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 18:00:25.285020 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282492 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282496 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282501 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282504 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282507 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282510 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282513 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282516 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282518 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282521 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282524 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282527 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282529 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282531 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282534 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282536 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282538 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282542 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282544 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 18:00:25.285397 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282547 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282549 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282552 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282554 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282557 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282560 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282569 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282572 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282575 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282577 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282580 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282583 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282585 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282588 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282590 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282592 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282595 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282597 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282600 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282602 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 18:00:25.285857 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282605 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282607 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282610 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282613 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282615 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282618 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282621 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282623 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282626 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282628 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282631 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282633 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282635 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282638 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282640 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282642 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282650 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282653 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282656 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282664 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 18:00:25.286334 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282667 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282669 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282672 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282674 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282677 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282679 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282681 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282684 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282686 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282688 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282691 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282693 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282696 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282698 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282700 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282703 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282706 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282708 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282712 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 18:00:25.286854 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282715 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282718 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282723 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282725 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282728 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282730 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282733 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282735 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282737 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282740 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282742 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282745 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282747 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282755 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282758 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282760 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 18:00:25.287342 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282762 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.282768 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282916 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282920 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282923 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282926 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282929 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282932 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282934 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282937 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282939 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282942 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282944 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282947 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282949 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 18:00:25.287817 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282952 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282954 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282956 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282959 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282963 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282966 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282969 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282973 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282975 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282978 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282980 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282982 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282985 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282987 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282990 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.282999 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283002 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283004 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283006 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 18:00:25.288247 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283008 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283011 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283013 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283015 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283017 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283020 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283022 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283024 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283027 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283029 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283031 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283034 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283036 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283038 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283041 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283043 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283046 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283048 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283050 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283053 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 18:00:25.288731 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283055 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283057 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283060 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283063 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283065 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283068 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283071 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283073 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283075 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283085 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283088 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283091 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283094 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283096 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283099 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283101 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283104 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283106 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283109 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 18:00:25.289502 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283111 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283114 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283116 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283118 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283121 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283123 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283126 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283128 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283130 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283133 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283135 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283138 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283140 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283143 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283145 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283147 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283150 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283152 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283154 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283156 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 18:00:25.290278 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283159 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283161 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283164 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283171 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283174 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283176 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283178 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:25.283181 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.283185 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.283301 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.286264 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.287596 2560 server.go:1019] "Starting client certificate rotation" Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.287760 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 18:00:25.290870 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.287811 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 18:00:25.291368 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.290396 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 18:00:25.294150 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.294126 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 18:00:25.301135 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.301118 2560 log.go:25] "Validated CRI v1 runtime API" Jul 21 18:00:25.304080 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.304065 2560 log.go:25] "Validated CRI v1 image API" Jul 21 18:00:25.304887 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.304872 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 18:00:25.307480 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.307459 2560 fs.go:135] Filesystem UUIDs: map[74f252bd-6b70-4aa3-9c37-0cd148c5de23:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 80c38436-1b89-4fd8-84df-bae0bdccc1cc:/dev/nvme0n1p3] Jul 21 18:00:25.307559 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.307478 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 18:00:25.311637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.311524 2560 manager.go:216] Machine: {Timestamp:2026-07-21 18:00:25.311049557 +0000 UTC m=+0.383645247 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093938 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c4b80429be1f3beccfa1b208c74fa SystemUUID:ec2c4b80-429b-e1f3-becc-fa1b208c74fa BootID:98a13760-3886-45b9-941e-99d05d342b11 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f2:07:2a:1a:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f2:07:2a:1a:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:6b:1b:dc:d2:21 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 18:00:25.311637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.311630 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 18:00:25.311736 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.311701 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 18:00:25.315713 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.315688 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 18:00:25.315865 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.315717 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-4.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 18:00:25.316467 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.316457 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 18:00:25.316510 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.316469 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 18:00:25.316510 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.316476 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 18:00:25.317348 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.317337 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 18:00:25.318574 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.318564 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 18:00:25.322335 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.322323 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 21 18:00:25.322484 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.322474 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 18:00:25.323272 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323263 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 21 18:00:25.323324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323276 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 18:00:25.323324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323299 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 18:00:25.323324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323309 2560 kubelet.go:398] "Adding apiserver pod source" Jul 21 18:00:25.323324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323322 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 18:00:25.323640 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323630 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 18:00:25.323689 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.323647 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 18:00:25.325950 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.325920 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 18:00:25.327254 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.327241 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 18:00:25.327309 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.327271 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 18:00:25.329246 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329234 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 18:00:25.329286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329251 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 18:00:25.329286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329258 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 18:00:25.329286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329263 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 18:00:25.329286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329273 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 18:00:25.329286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329278 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 18:00:25.329587 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329578 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 18:00:25.329623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329588 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 18:00:25.329623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329596 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 18:00:25.329623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329601 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 18:00:25.329623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329615 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 18:00:25.329623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.329622 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 18:00:25.331249 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.331239 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 18:00:25.331249 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.331249 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 18:00:25.334244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.334232 2560 server.go:1268] "Started kubelet" Jul 21 18:00:25.334420 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.334317 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 18:00:25.334533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.334324 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 18:00:25.334533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.334437 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 18:00:25.335080 ip-10-0-139-4 systemd[1]: Started Kubernetes Kubelet. Jul 21 18:00:25.335704 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.335579 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 18:00:25.340295 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.340274 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 21 18:00:25.340812 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.340780 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 18:00:25.344847 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.344826 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-4.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 18:00:25.344928 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.344902 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 18:00:25.344970 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.344923 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-4.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 18:00:25.346161 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.345125 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-4.ec2.internal.18c45fed5805896f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-4.ec2.internal,UID:ip-10-0-139-4.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-4.ec2.internal,},FirstTimestamp:2026-07-21 18:00:25.334204783 +0000 UTC m=+0.406800472,LastTimestamp:2026-07-21 18:00:25.334204783 +0000 UTC m=+0.406800472,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-4.ec2.internal,}" Jul 21 18:00:25.346351 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.346338 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 18:00:25.346398 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.346384 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 18:00:25.346695 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.346462 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:25.347021 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347005 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 18:00:25.347021 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347006 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 18:00:25.347159 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347031 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 18:00:25.347159 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347087 2560 factory.go:55] Registering systemd factory Jul 21 18:00:25.347159 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347101 2560 factory.go:223] Registration of the systemd container factory successfully Jul 21 18:00:25.347159 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347142 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 21 18:00:25.347159 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347149 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 21 18:00:25.347372 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347309 2560 factory.go:153] Registering CRI-O factory Jul 21 18:00:25.347372 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347322 2560 factory.go:223] Registration of the crio container factory successfully Jul 21 18:00:25.347498 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347379 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 18:00:25.347498 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347404 2560 factory.go:103] Registering Raw factory Jul 21 18:00:25.347498 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.347419 2560 manager.go:1196] Started watching for new ooms in manager Jul 21 18:00:25.348235 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.348122 2560 manager.go:349] Starting recovery of all containers Jul 21 18:00:25.353940 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.353911 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-4.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 18:00:25.354069 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.354031 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 18:00:25.354492 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.354296 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 18:00:25.358279 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.358171 2560 manager.go:354] Recovery completed Jul 21 18:00:25.360077 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.360050 2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 21 18:00:25.362926 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.362913 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.365736 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.365721 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.365806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.365750 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.365806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.365760 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.366178 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366164 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 18:00:25.366178 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366178 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 18:00:25.366276 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366184 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.366276 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366195 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 21 18:00:25.366881 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366865 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.366958 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366891 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.366958 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.366900 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.367536 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.367524 2560 policy_none.go:49] "None policy: Start" Jul 21 18:00:25.367580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.367541 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 18:00:25.367580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.367551 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 18:00:25.368763 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.368753 2560 policy_none.go:47] "Start" Jul 21 18:00:25.369099 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.369013 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-4.ec2.internal.18c45fed59e6ac3d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-4.ec2.internal,UID:ip-10-0-139-4.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-4.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-4.ec2.internal,},FirstTimestamp:2026-07-21 18:00:25.365736509 +0000 UTC m=+0.438332198,LastTimestamp:2026-07-21 18:00:25.365736509 +0000 UTC m=+0.438332198,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-4.ec2.internal,}" Jul 21 18:00:25.392413 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.392354 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-4.ec2.internal.18c45fed59e6f3ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-4.ec2.internal,UID:ip-10-0-139-4.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-4.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-4.ec2.internal,},FirstTimestamp:2026-07-21 18:00:25.365754797 +0000 UTC m=+0.438350486,LastTimestamp:2026-07-21 18:00:25.365754797 +0000 UTC m=+0.438350486,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-4.ec2.internal,}" Jul 21 18:00:25.409004 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.408991 2560 manager.go:337] "Starting Device Plugin manager" Jul 21 18:00:25.409091 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.409023 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 18:00:25.409091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.409032 2560 server.go:87] "Starting device plugin registration server" Jul 21 18:00:25.409272 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.409260 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 18:00:25.409338 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.409275 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 18:00:25.409391 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.409359 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 18:00:25.409520 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.409459 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 18:00:25.409520 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.409471 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 18:00:25.409987 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.409967 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 18:00:25.410055 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.410023 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:25.412646 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.412587 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-4.ec2.internal.18c45fed59e715bb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-4.ec2.internal,UID:ip-10-0-139-4.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-4.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-4.ec2.internal,},FirstTimestamp:2026-07-21 18:00:25.365763515 +0000 UTC m=+0.438359204,LastTimestamp:2026-07-21 18:00:25.365763515 +0000 UTC m=+0.438359204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-4.ec2.internal,}" Jul 21 18:00:25.427934 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.427916 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbqmm" Jul 21 18:00:25.428367 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.428302 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-4.ec2.internal.18c45fed59e6ac3d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-4.ec2.internal.18c45fed59e6ac3d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-4.ec2.internal,UID:ip-10-0-139-4.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-4.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-4.ec2.internal,},FirstTimestamp:2026-07-21 18:00:25.365736509 +0000 UTC m=+0.438332198,LastTimestamp:2026-07-21 18:00:25.366879676 +0000 UTC m=+0.439475365,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-4.ec2.internal,}" Jul 21 18:00:25.450609 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.450587 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbqmm" Jul 21 18:00:25.454684 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.451719 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-4.ec2.internal.18c45fed59e6f3ad\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-4.ec2.internal.18c45fed59e6f3ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-4.ec2.internal,UID:ip-10-0-139-4.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-4.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-4.ec2.internal,},FirstTimestamp:2026-07-21 18:00:25.365754797 +0000 UTC m=+0.438350486,LastTimestamp:2026-07-21 18:00:25.366895089 +0000 UTC m=+0.439490781,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-4.ec2.internal,}" Jul 21 18:00:25.510265 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.510237 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.511286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.511268 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.511418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.511296 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.511418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.511306 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.511418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.511331 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.525163 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.525081 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 18:00:25.526515 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.526489 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 18:00:25.526609 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.526541 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 18:00:25.526609 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.526568 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 18:00:25.526689 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.526607 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 18:00:25.526756 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.526736 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.526864 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.526774 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-4.ec2.internal\": node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:25.543137 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.543118 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:25.626908 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.626873 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal"] Jul 21 18:00:25.627058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.626945 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.628543 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.628515 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.628627 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.628547 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.628627 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.628562 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.629830 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.629817 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.629960 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.629942 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.630019 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.629985 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.630519 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.630502 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.630605 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.630531 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.630605 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.630540 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.630605 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.630502 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.630708 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.630625 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.630708 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.630638 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.631947 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.631933 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.631996 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.631958 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:25.632609 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.632591 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:25.632686 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.632624 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:25.632686 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.632640 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:25.648892 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.648866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc5664b47cc83a1d98ee9d63320a826b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal\" (UID: \"fc5664b47cc83a1d98ee9d63320a826b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.648892 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.648899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc5664b47cc83a1d98ee9d63320a826b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal\" (UID: \"fc5664b47cc83a1d98ee9d63320a826b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.649013 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.648914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/143a5776505ae49b336b0210346bca91-config\") pod \"kube-apiserver-proxy-ip-10-0-139-4.ec2.internal\" (UID: \"143a5776505ae49b336b0210346bca91\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.660020 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.659997 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-4.ec2.internal\" not found" node="ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.663842 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.663826 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-4.ec2.internal\" not found" node="ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.674469 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.674435 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:25.749768 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.749737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc5664b47cc83a1d98ee9d63320a826b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal\" (UID: \"fc5664b47cc83a1d98ee9d63320a826b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.749884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.749785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc5664b47cc83a1d98ee9d63320a826b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal\" (UID: \"fc5664b47cc83a1d98ee9d63320a826b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.749884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.749810 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/143a5776505ae49b336b0210346bca91-config\") pod \"kube-apiserver-proxy-ip-10-0-139-4.ec2.internal\" (UID: \"143a5776505ae49b336b0210346bca91\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.749884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.749851 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc5664b47cc83a1d98ee9d63320a826b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal\" (UID: \"fc5664b47cc83a1d98ee9d63320a826b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.749884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.749862 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/143a5776505ae49b336b0210346bca91-config\") pod \"kube-apiserver-proxy-ip-10-0-139-4.ec2.internal\" (UID: \"143a5776505ae49b336b0210346bca91\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.750026 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.749856 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc5664b47cc83a1d98ee9d63320a826b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal\" (UID: \"fc5664b47cc83a1d98ee9d63320a826b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.774881 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.774856 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:25.875788 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.875768 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:25.961567 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.961537 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.967759 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:25.967740 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" Jul 21 18:00:25.976665 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:25.976645 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.078584 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:26.078546 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.178730 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:26.178657 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.279733 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:26.279684 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.287452 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.287428 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 18:00:26.287641 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.287621 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 18:00:26.380009 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:26.379974 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.409355 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.409324 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:26.458402 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.458318 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 17:55:25 +0000 UTC" deadline="2027-12-24 19:40:53.901402256 +0000 UTC" Jul 21 18:00:26.458402 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.458349 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12505h40m27.443056416s" Jul 21 18:00:26.481152 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:26.481114 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.492373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.492355 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:26.493335 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.493309 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:26.493490 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.493347 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:26.493490 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.493362 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:26.582092 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:26.582060 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-4.ec2.internal\" not found" Jul 21 18:00:26.635749 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:26.635698 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143a5776505ae49b336b0210346bca91.slice/crio-ef12f3f98839129fb3a1c3f554eaef88572549f14f6f26fe4eb291e70979036a WatchSource:0}: Error finding container ef12f3f98839129fb3a1c3f554eaef88572549f14f6f26fe4eb291e70979036a: Status 404 returned error can't find the container with id ef12f3f98839129fb3a1c3f554eaef88572549f14f6f26fe4eb291e70979036a Jul 21 18:00:26.636318 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:26.636295 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5664b47cc83a1d98ee9d63320a826b.slice/crio-c6458431b90702c1238c6973d46aef2852f91fab85d77d82c17341ae7e43556e WatchSource:0}: Error finding container c6458431b90702c1238c6973d46aef2852f91fab85d77d82c17341ae7e43556e: Status 404 returned error can't find the container with id c6458431b90702c1238c6973d46aef2852f91fab85d77d82c17341ae7e43556e Jul 21 18:00:26.640888 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.640871 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:00:26.679904 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.679876 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:26.747515 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.747432 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" Jul 21 18:00:26.765517 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.765496 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 18:00:26.768016 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.768001 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" Jul 21 18:00:26.783456 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.783437 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 18:00:26.863181 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:26.863140 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:27.327716 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.327686 2560 apiserver.go:52] "Watching apiserver" Jul 21 18:00:27.342088 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.342065 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 18:00:27.344276 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.344252 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-f85qv","openshift-multus/network-metrics-daemon-ltwt9","openshift-network-operator/iptables-alerter-8j6h9","openshift-ovn-kubernetes/ovnkube-node-jzqwp","kube-system/global-pull-secret-syncer-kzvc2","kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc","openshift-dns/node-resolver-rvrzw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal","openshift-multus/multus-additional-cni-plugins-dcrrq","openshift-network-diagnostics/network-check-target-2cnsl","kube-system/konnectivity-agent-q64wj","openshift-cluster-node-tuning-operator/tuned-cxnkr","openshift-image-registry/node-ca-bf96n"] Jul 21 18:00:27.346547 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.346528 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.346615 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.346606 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 18:00:27.348306 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.348291 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:27.348388 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.348363 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:27.349426 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.349408 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.349664 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.349643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 18:00:27.349822 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.349804 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.349975 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.349954 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.350470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.350458 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqkb7\"" Jul 21 18:00:27.350971 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.350958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.351510 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.351496 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.352217 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.352203 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.353509 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.353492 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.354932 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.354916 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.355639 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.355623 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.356112 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.356097 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357149 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357204 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357311 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357396 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-registration-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357589 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q896n\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-device-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357662 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357679 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357748 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-socket-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-sys-fs\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357844 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357867 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk6mb\" (UniqueName: \"kubernetes.io/projected/7481ab0b-595c-4540-8b21-abad89e0aed6-kube-api-access-dk6mb\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.357919 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.358110 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bscjt\"" Jul 21 18:00:27.358301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.358284 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 18:00:27.359187 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.358640 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 18:00:27.359187 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.358895 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.359187 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.358984 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8n87w\"" Jul 21 18:00:27.359187 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359183 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zmp9x\"" Jul 21 18:00:27.359366 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359248 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 18:00:27.359366 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359352 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.359489 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359363 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 18:00:27.359546 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359507 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 18:00:27.359596 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359556 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.359686 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359661 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tkz44\"" Jul 21 18:00:27.359902 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.359874 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 18:00:27.360105 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.360088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:27.360187 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.360097 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sh8dg\"" Jul 21 18:00:27.360187 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.360174 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:27.361676 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.361662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.363035 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.363022 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.363991 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.363971 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.364175 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.364162 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.364271 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.364253 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 18:00:27.364325 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.364289 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r4mhs\"" Jul 21 18:00:27.364482 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.364439 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.364576 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.364517 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:27.365199 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.365182 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 18:00:27.365663 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.365646 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7dnrj\"" Jul 21 18:00:27.365747 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.365683 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 18:00:27.366666 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.366650 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 18:00:27.411015 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.410949 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c4gml" Jul 21 18:00:27.420604 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.420584 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c4gml" Jul 21 18:00:27.448011 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.447991 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 18:00:27.459068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-device-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459168 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459077 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-multus-certs\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459168 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-system-cni-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459168 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459124 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-os-release\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459168 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459149 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-device-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459331 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459166 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.459331 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-modprobe-d\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.459331 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459331 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459289 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15cb2308-3ce4-450f-a0c9-4c391412109b-host\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.459331 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/964214a8-2f78-4f76-bd48-dc273435e190-cni-binary-copy\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459339 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-cni-bin\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-hostroot\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8878bc1b-0748-4045-b076-30860a1b7db4-host-slash\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459461 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-os-release\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-host\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459504 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-systemd\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-var-lib-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-etc-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.459557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovnkube-config\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15cb2308-3ce4-450f-a0c9-4c391412109b-serviceca\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-tmp-dir\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-systemd\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovn-node-metrics-cert\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459654 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-netns\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459685 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-lib-modules\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e01f450-fbc9-45f0-8d02-287ecc687cee-tmp\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-socket-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-sys-fs\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk6mb\" (UniqueName: \"kubernetes.io/projected/7481ab0b-595c-4540-8b21-abad89e0aed6-kube-api-access-dk6mb\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-ovn\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9frk\" (UniqueName: \"kubernetes.io/projected/15cb2308-3ce4-450f-a0c9-4c391412109b-kube-api-access-w9frk\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459839 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-socket-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07542b9-a38f-4338-8602-60373dc3d93a-konnectivity-ca\") pod \"konnectivity-agent-q64wj\" (UID: \"e07542b9-a38f-4338-8602-60373dc3d93a\") " pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459802 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-sys-fs\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.459872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysconfig\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-sys\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459934 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-log-socket\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-cni-netd\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.459986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-env-overrides\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460035 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07542b9-a38f-4338-8602-60373dc3d93a-agent-certs\") pod \"konnectivity-agent-q64wj\" (UID: \"e07542b9-a38f-4338-8602-60373dc3d93a\") " pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460058 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-cnibin\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5m7q\" (UniqueName: \"kubernetes.io/projected/964214a8-2f78-4f76-bd48-dc273435e190-kube-api-access-v5m7q\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460110 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-system-cni-dir\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-cni-bin\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-socket-dir-parent\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cnibin\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysctl-d\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.460637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460257 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-var-lib-kubelet\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460294 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffqvk\" (UniqueName: \"kubernetes.io/projected/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-kube-api-access-ffqvk\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460343 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-registration-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovnkube-script-lib\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz9bs\" (UniqueName: \"kubernetes.io/projected/5acea20a-96d0-4ebf-b6be-27366b1575ce-kube-api-access-wz9bs\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-k8s-cni-cncf-io\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7481ab0b-595c-4540-8b21-abad89e0aed6-registration-dir\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-conf-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460499 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-kubernetes\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-slash\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460548 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz625\" (UniqueName: \"kubernetes.io/projected/8878bc1b-0748-4045-b076-30860a1b7db4-kube-api-access-vz625\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd5k6\" (UniqueName: \"kubernetes.io/projected/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-kube-api-access-vd5k6\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460599 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-tuned\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460633 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-kubelet\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460663 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysctl-conf\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7bl9\" (UniqueName: \"kubernetes.io/projected/225d56ab-f30e-44dc-bc64-42071d0e90b8-kube-api-access-j7bl9\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460750 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-run\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-run-netns\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8878bc1b-0748-4045-b076-30860a1b7db4-iptables-alerter-script\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad0416fc-678d-4d6d-8715-24d76b208eb5-dbus\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-cni-multus\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460834 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-kubelet\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-etc-kubernetes\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-479fr\" (UniqueName: \"kubernetes.io/projected/1e01f450-fbc9-45f0-8d02-287ecc687cee-kube-api-access-479fr\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-systemd-units\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.461633 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460934 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-node-log\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.462091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460954 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-cni-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.462091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460976 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/964214a8-2f78-4f76-bd48-dc273435e190-multus-daemon-config\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.462091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.460995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad0416fc-678d-4d6d-8715-24d76b208eb5-kubelet-config\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.462091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.461018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-hosts-file\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.467674 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.467659 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:27.467716 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.467679 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:27.467716 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.467688 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:27.467798 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.467748 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:27.967733779 +0000 UTC m=+3.040329474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:27.468043 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.468015 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 18:00:27.471073 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.471054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk6mb\" (UniqueName: \"kubernetes.io/projected/7481ab0b-595c-4540-8b21-abad89e0aed6-kube-api-access-dk6mb\") pod \"aws-ebs-csi-driver-node-7j2vc\" (UID: \"7481ab0b-595c-4540-8b21-abad89e0aed6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.533028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.532985 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" event={"ID":"143a5776505ae49b336b0210346bca91","Type":"ContainerStarted","Data":"ef12f3f98839129fb3a1c3f554eaef88572549f14f6f26fe4eb291e70979036a"} Jul 21 18:00:27.534024 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.534002 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" event={"ID":"fc5664b47cc83a1d98ee9d63320a826b","Type":"ContainerStarted","Data":"c6458431b90702c1238c6973d46aef2852f91fab85d77d82c17341ae7e43556e"} Jul 21 18:00:27.561625 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15cb2308-3ce4-450f-a0c9-4c391412109b-host\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.561704 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/964214a8-2f78-4f76-bd48-dc273435e190-cni-binary-copy\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.561704 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-cni-bin\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.561704 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-hostroot\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.561704 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8878bc1b-0748-4045-b076-30860a1b7db4-host-slash\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.561854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15cb2308-3ce4-450f-a0c9-4c391412109b-host\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.561854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-cni-bin\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.561854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8878bc1b-0748-4045-b076-30860a1b7db4-host-slash\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.561854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561794 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-hostroot\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.561854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-os-release\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-host\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-systemd\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561902 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-os-release\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-var-lib-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-host\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-etc-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561964 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-systemd\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovnkube-config\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.561997 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-var-lib-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562015 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15cb2308-3ce4-450f-a0c9-4c391412109b-serviceca\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562013 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-etc-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.562041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-tmp-dir\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-systemd\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovn-node-metrics-cert\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562102 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-netns\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562128 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-lib-modules\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e01f450-fbc9-45f0-8d02-287ecc687cee-tmp\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-ovn\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9frk\" (UniqueName: \"kubernetes.io/projected/15cb2308-3ce4-450f-a0c9-4c391412109b-kube-api-access-w9frk\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07542b9-a38f-4338-8602-60373dc3d93a-konnectivity-ca\") pod \"konnectivity-agent-q64wj\" (UID: \"e07542b9-a38f-4338-8602-60373dc3d93a\") " pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysconfig\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-sys\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-log-socket\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-cni-netd\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562370 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-env-overrides\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07542b9-a38f-4338-8602-60373dc3d93a-agent-certs\") pod \"konnectivity-agent-q64wj\" (UID: \"e07542b9-a38f-4338-8602-60373dc3d93a\") " pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-cnibin\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.564357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562476 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-tmp-dir\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5m7q\" (UniqueName: \"kubernetes.io/projected/964214a8-2f78-4f76-bd48-dc273435e190-kube-api-access-v5m7q\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562151 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-systemd\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-system-cni-dir\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-cni-bin\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15cb2308-3ce4-450f-a0c9-4c391412109b-serviceca\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovnkube-config\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562585 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-ovn\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562616 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-lib-modules\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562625 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-socket-dir-parent\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cnibin\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562666 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-netns\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562680 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysctl-d\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-var-lib-kubelet\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-socket-dir-parent\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffqvk\" (UniqueName: \"kubernetes.io/projected/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-kube-api-access-ffqvk\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovnkube-script-lib\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.566470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562751 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562757 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz9bs\" (UniqueName: \"kubernetes.io/projected/5acea20a-96d0-4ebf-b6be-27366b1575ce-kube-api-access-wz9bs\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562764 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-k8s-cni-cncf-io\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-conf-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562837 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-var-lib-kubelet\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-kubernetes\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562883 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cnibin\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562887 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-slash\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562918 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-slash\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz625\" (UniqueName: \"kubernetes.io/projected/8878bc1b-0748-4045-b076-30860a1b7db4-kube-api-access-vz625\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562959 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-log-socket\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562964 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysctl-d\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/964214a8-2f78-4f76-bd48-dc273435e190-cni-binary-copy\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-cni-netd\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563079 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e07542b9-a38f-4338-8602-60373dc3d93a-konnectivity-ca\") pod \"konnectivity-agent-q64wj\" (UID: \"e07542b9-a38f-4338-8602-60373dc3d93a\") " pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.567286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.562965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563146 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-k8s-cni-cncf-io\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563185 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-conf-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-kubernetes\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563241 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysconfig\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd5k6\" (UniqueName: \"kubernetes.io/projected/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-kube-api-access-vd5k6\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563278 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-tuned\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563491 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-env-overrides\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563532 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovnkube-script-lib\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-cni-bin\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563565 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-system-cni-dir\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-sys\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563598 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-kubelet\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563619 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-cnibin\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.563651 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563661 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-kubelet\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568120 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563666 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-run-openvswitch\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563711 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysctl-conf\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.563731 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:28.063713191 +0000 UTC m=+3.136308887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563819 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-sysctl-conf\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7bl9\" (UniqueName: \"kubernetes.io/projected/225d56ab-f30e-44dc-bc64-42071d0e90b8-kube-api-access-j7bl9\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563932 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-run\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-run-netns\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.563981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8878bc1b-0748-4045-b076-30860a1b7db4-iptables-alerter-script\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad0416fc-678d-4d6d-8715-24d76b208eb5-dbus\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564024 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-run\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-cni-multus\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564066 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-cni-multus\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.568765 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-kubelet\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564102 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-host-run-netns\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-etc-kubernetes\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-479fr\" (UniqueName: \"kubernetes.io/projected/1e01f450-fbc9-45f0-8d02-287ecc687cee-kube-api-access-479fr\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-systemd-units\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-node-log\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-cni-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.564281 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.564337 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:28.064320352 +0000 UTC m=+3.136916031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/964214a8-2f78-4f76-bd48-dc273435e190-multus-daemon-config\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad0416fc-678d-4d6d-8715-24d76b208eb5-kubelet-config\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-hosts-file\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564481 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-multus-certs\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-system-cni-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-os-release\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564558 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-modprobe-d\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.569290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8878bc1b-0748-4045-b076-30860a1b7db4-iptables-alerter-script\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564705 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-etc-kubernetes\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564711 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-modprobe-d\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.564767 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-multus-cni-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad0416fc-678d-4d6d-8715-24d76b208eb5-dbus\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565128 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565185 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-var-lib-kubelet\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565202 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/964214a8-2f78-4f76-bd48-dc273435e190-multus-daemon-config\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-host-run-multus-certs\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565268 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-system-cni-dir\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565277 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad0416fc-678d-4d6d-8715-24d76b208eb5-kubelet-config\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565327 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-hosts-file\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565330 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/964214a8-2f78-4f76-bd48-dc273435e190-os-release\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565329 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5acea20a-96d0-4ebf-b6be-27366b1575ce-ovn-node-metrics-cert\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565388 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-systemd-units\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5acea20a-96d0-4ebf-b6be-27366b1575ce-node-log\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.565758 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-cni-binary-copy\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.566316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e01f450-fbc9-45f0-8d02-287ecc687cee-etc-tuned\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.569884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.567324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e07542b9-a38f-4338-8602-60373dc3d93a-agent-certs\") pod \"konnectivity-agent-q64wj\" (UID: \"e07542b9-a38f-4338-8602-60373dc3d93a\") " pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.570418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.569582 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e01f450-fbc9-45f0-8d02-287ecc687cee-tmp\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.581984 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.581196 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd5k6\" (UniqueName: \"kubernetes.io/projected/0fdef49f-195a-4d8f-9e49-6fbb46a08a84-kube-api-access-vd5k6\") pod \"multus-additional-cni-plugins-dcrrq\" (UID: \"0fdef49f-195a-4d8f-9e49-6fbb46a08a84\") " pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.582532 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.582434 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz9bs\" (UniqueName: \"kubernetes.io/projected/5acea20a-96d0-4ebf-b6be-27366b1575ce-kube-api-access-wz9bs\") pod \"ovnkube-node-jzqwp\" (UID: \"5acea20a-96d0-4ebf-b6be-27366b1575ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.582863 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.582841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffqvk\" (UniqueName: \"kubernetes.io/projected/e85ae40c-e1d2-45cd-87c5-4c63e711ef78-kube-api-access-ffqvk\") pod \"node-resolver-rvrzw\" (UID: \"e85ae40c-e1d2-45cd-87c5-4c63e711ef78\") " pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.583077 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.583059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9frk\" (UniqueName: \"kubernetes.io/projected/15cb2308-3ce4-450f-a0c9-4c391412109b-kube-api-access-w9frk\") pod \"node-ca-bf96n\" (UID: \"15cb2308-3ce4-450f-a0c9-4c391412109b\") " pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.583886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.583825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz625\" (UniqueName: \"kubernetes.io/projected/8878bc1b-0748-4045-b076-30860a1b7db4-kube-api-access-vz625\") pod \"iptables-alerter-8j6h9\" (UID: \"8878bc1b-0748-4045-b076-30860a1b7db4\") " pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.583886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.583844 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5m7q\" (UniqueName: \"kubernetes.io/projected/964214a8-2f78-4f76-bd48-dc273435e190-kube-api-access-v5m7q\") pod \"multus-f85qv\" (UID: \"964214a8-2f78-4f76-bd48-dc273435e190\") " pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.584727 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.584709 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-479fr\" (UniqueName: \"kubernetes.io/projected/1e01f450-fbc9-45f0-8d02-287ecc687cee-kube-api-access-479fr\") pod \"tuned-cxnkr\" (UID: \"1e01f450-fbc9-45f0-8d02-287ecc687cee\") " pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.585478 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.585434 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7bl9\" (UniqueName: \"kubernetes.io/projected/225d56ab-f30e-44dc-bc64-42071d0e90b8-kube-api-access-j7bl9\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:27.656566 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.656531 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" Jul 21 18:00:27.665749 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.665718 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8j6h9" Jul 21 18:00:27.667328 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.667299 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7481ab0b_595c_4540_8b21_abad89e0aed6.slice/crio-1712a04ad581222407261c33885b7cddf2ea7794056992bd2d3b7ead774e4533 WatchSource:0}: Error finding container 1712a04ad581222407261c33885b7cddf2ea7794056992bd2d3b7ead774e4533: Status 404 returned error can't find the container with id 1712a04ad581222407261c33885b7cddf2ea7794056992bd2d3b7ead774e4533 Jul 21 18:00:27.672499 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.672479 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:27.676474 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.676425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bf96n" Jul 21 18:00:27.679166 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.679138 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8878bc1b_0748_4045_b076_30860a1b7db4.slice/crio-e67d7b388300ece8e281ee376d691c20c70dab2a9f4c274fc8d3311866859d54 WatchSource:0}: Error finding container e67d7b388300ece8e281ee376d691c20c70dab2a9f4c274fc8d3311866859d54: Status 404 returned error can't find the container with id e67d7b388300ece8e281ee376d691c20c70dab2a9f4c274fc8d3311866859d54 Jul 21 18:00:27.681370 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.681350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:27.683812 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.683786 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5acea20a_96d0_4ebf_b6be_27366b1575ce.slice/crio-b2fbcb677d0a34a4ee01f9d26476310eb4b04aef3d0826d10187e6ba06fea690 WatchSource:0}: Error finding container b2fbcb677d0a34a4ee01f9d26476310eb4b04aef3d0826d10187e6ba06fea690: Status 404 returned error can't find the container with id b2fbcb677d0a34a4ee01f9d26476310eb4b04aef3d0826d10187e6ba06fea690 Jul 21 18:00:27.686303 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.686285 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" Jul 21 18:00:27.691069 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.691043 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07542b9_a38f_4338_8602_60373dc3d93a.slice/crio-530d8ff0ede2027b589b5ef1c7077c30fb3944481f420e87d9808658dd7253fc WatchSource:0}: Error finding container 530d8ff0ede2027b589b5ef1c7077c30fb3944481f420e87d9808658dd7253fc: Status 404 returned error can't find the container with id 530d8ff0ede2027b589b5ef1c7077c30fb3944481f420e87d9808658dd7253fc Jul 21 18:00:27.696354 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.696333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" Jul 21 18:00:27.696823 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.696800 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fdef49f_195a_4d8f_9e49_6fbb46a08a84.slice/crio-f765d6d2dd717545d7291c6b8d362681aeae0eb8d28fc0b8a04903ccc20d4ce6 WatchSource:0}: Error finding container f765d6d2dd717545d7291c6b8d362681aeae0eb8d28fc0b8a04903ccc20d4ce6: Status 404 returned error can't find the container with id f765d6d2dd717545d7291c6b8d362681aeae0eb8d28fc0b8a04903ccc20d4ce6 Jul 21 18:00:27.705155 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.705092 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15cb2308_3ce4_450f_a0c9_4c391412109b.slice/crio-a9cc51a206f6ab54ed7785246b9c91a3a01121a394a7e744bb44e5cf5373e725 WatchSource:0}: Error finding container a9cc51a206f6ab54ed7785246b9c91a3a01121a394a7e744bb44e5cf5373e725: Status 404 returned error can't find the container with id a9cc51a206f6ab54ed7785246b9c91a3a01121a394a7e744bb44e5cf5373e725 Jul 21 18:00:27.707006 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.706972 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e01f450_fbc9_45f0_8d02_287ecc687cee.slice/crio-4626c19222a406d37f92fca75898658a92f64a2179e900e460060d7c3a7a731d WatchSource:0}: Error finding container 4626c19222a406d37f92fca75898658a92f64a2179e900e460060d7c3a7a731d: Status 404 returned error can't find the container with id 4626c19222a406d37f92fca75898658a92f64a2179e900e460060d7c3a7a731d Jul 21 18:00:27.708882 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.708851 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f85qv" Jul 21 18:00:27.713213 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.713191 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rvrzw" Jul 21 18:00:27.719849 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:00:27.719827 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964214a8_2f78_4f76_bd48_dc273435e190.slice/crio-24e3c574cb5c3b126add5a5b25c2f24330db81f94cf3878f01f0d473f9541f17 WatchSource:0}: Error finding container 24e3c574cb5c3b126add5a5b25c2f24330db81f94cf3878f01f0d473f9541f17: Status 404 returned error can't find the container with id 24e3c574cb5c3b126add5a5b25c2f24330db81f94cf3878f01f0d473f9541f17 Jul 21 18:00:27.738303 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.738279 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:27.970176 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:27.969585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:27.970176 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.969747 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:27.970176 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.969766 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:27.970176 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.969779 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:27.970176 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:27.969838 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:28.96981958 +0000 UTC m=+4.042415259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:28.070810 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.070761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:28.070810 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.070815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:28.071008 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.070940 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:28.071008 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.071003 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:29.070985308 +0000 UTC m=+4.143580988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:28.071395 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.071374 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:28.071479 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.071434 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:29.07141744 +0000 UTC m=+4.144013123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:28.423304 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.423243 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:55:27 +0000 UTC" deadline="2027-12-19 08:07:29.121837221 +0000 UTC" Jul 21 18:00:28.423304 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.423303 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12374h7m0.698538791s" Jul 21 18:00:28.536915 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.536878 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" event={"ID":"1e01f450-fbc9-45f0-8d02-287ecc687cee","Type":"ContainerStarted","Data":"4626c19222a406d37f92fca75898658a92f64a2179e900e460060d7c3a7a731d"} Jul 21 18:00:28.538653 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.538617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q64wj" event={"ID":"e07542b9-a38f-4338-8602-60373dc3d93a","Type":"ContainerStarted","Data":"530d8ff0ede2027b589b5ef1c7077c30fb3944481f420e87d9808658dd7253fc"} Jul 21 18:00:28.540351 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.540313 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8j6h9" event={"ID":"8878bc1b-0748-4045-b076-30860a1b7db4","Type":"ContainerStarted","Data":"e67d7b388300ece8e281ee376d691c20c70dab2a9f4c274fc8d3311866859d54"} Jul 21 18:00:28.541861 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.541834 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f85qv" event={"ID":"964214a8-2f78-4f76-bd48-dc273435e190","Type":"ContainerStarted","Data":"24e3c574cb5c3b126add5a5b25c2f24330db81f94cf3878f01f0d473f9541f17"} Jul 21 18:00:28.543347 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.543318 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerStarted","Data":"f765d6d2dd717545d7291c6b8d362681aeae0eb8d28fc0b8a04903ccc20d4ce6"} Jul 21 18:00:28.544492 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.544434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bf96n" event={"ID":"15cb2308-3ce4-450f-a0c9-4c391412109b","Type":"ContainerStarted","Data":"a9cc51a206f6ab54ed7785246b9c91a3a01121a394a7e744bb44e5cf5373e725"} Jul 21 18:00:28.545695 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.545658 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"b2fbcb677d0a34a4ee01f9d26476310eb4b04aef3d0826d10187e6ba06fea690"} Jul 21 18:00:28.547185 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.547159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" event={"ID":"7481ab0b-595c-4540-8b21-abad89e0aed6","Type":"ContainerStarted","Data":"1712a04ad581222407261c33885b7cddf2ea7794056992bd2d3b7ead774e4533"} Jul 21 18:00:28.548324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.548292 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rvrzw" event={"ID":"e85ae40c-e1d2-45cd-87c5-4c63e711ef78","Type":"ContainerStarted","Data":"0afc4a5a30ea6d8402f7c5362f8b19a182fadbc1dfd9fd1bbc3548f3b7120950"} Jul 21 18:00:28.978434 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:28.978389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:28.978632 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.978583 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:28.978632 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.978603 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:28.978632 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.978615 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:28.978793 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:28.978674 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:30.978655835 +0000 UTC m=+6.051251514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:29.079883 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.079134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:29.079883 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.079193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:29.079883 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.079338 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:29.079883 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.079401 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:31.079380406 +0000 UTC m=+6.151976113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:29.079883 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.079796 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:29.079883 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.079844 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:31.07982871 +0000 UTC m=+6.152424389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:29.423607 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.423561 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:55:27 +0000 UTC" deadline="2028-04-17 18:46:24.991789352 +0000 UTC" Jul 21 18:00:29.423607 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.423605 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15264h45m55.568188912s" Jul 21 18:00:29.530999 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.529889 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:29.530999 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.530037 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:29.530999 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.530433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:29.530999 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.530547 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:29.530999 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:29.530611 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:29.530999 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:29.530676 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:30.995121 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:30.994461 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:30.995121 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:30.994650 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:30.995121 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:30.994673 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:30.995121 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:30.994685 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:30.995121 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:30.994746 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:34.99472564 +0000 UTC m=+10.067321320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:31.096400 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:31.095663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:31.096400 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:31.095710 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:31.096400 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.095861 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:31.096400 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.095919 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.09590133 +0000 UTC m=+10.168497009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:31.096400 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.096312 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:31.096400 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.096367 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.096351665 +0000 UTC m=+10.168947346 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:31.527398 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:31.527364 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:31.527604 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.527513 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:31.527879 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:31.527862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:31.527964 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.527945 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:31.528086 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:31.528071 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:31.528188 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:31.528164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:33.529637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:33.528954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:33.529637 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:33.529069 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:33.529637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:33.529365 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:33.529637 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:33.529430 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:33.529637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:33.529489 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:33.529637 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:33.529561 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:35.032751 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:35.032691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:35.033303 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.032877 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:35.033303 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.032899 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:35.033303 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.033267 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:35.034536 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.034493 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.033324906 +0000 UTC m=+18.105920596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:35.133361 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:35.133318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:35.133361 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:35.133378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:35.133707 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.133497 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:35.133707 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.133514 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:35.133707 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.133582 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.133559994 +0000 UTC m=+18.206155684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:35.133707 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.133601 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.133592272 +0000 UTC m=+18.206187948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:35.527975 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:35.527864 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:35.528161 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.527987 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:35.528662 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:35.528272 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:35.528662 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:35.528382 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:35.528662 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.528504 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:35.528662 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:35.528633 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:37.526966 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:37.526929 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:37.527427 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:37.527081 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:37.527427 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:37.526939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:37.527427 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:37.527182 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:37.527427 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:37.526931 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:37.527427 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:37.527261 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:39.527160 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:39.527120 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:39.527638 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:39.527243 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:39.527638 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:39.527120 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:39.527638 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:39.527353 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:39.527638 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:39.527122 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:39.527638 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:39.527426 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:41.529637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:41.529605 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:41.530072 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:41.529604 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:41.530072 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:41.529747 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:41.530072 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:41.529604 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:41.530072 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:41.529787 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:41.530072 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:41.529885 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:43.091600 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:43.091508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:43.092037 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.091662 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:43.092037 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.091680 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:43.092037 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.091690 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:43.092037 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.091750 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:59.091731556 +0000 UTC m=+34.164327233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:43.192241 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:43.192206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:43.192241 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:43.192243 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:43.192483 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.192368 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:43.192483 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.192384 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:43.192483 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.192431 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:59.192413207 +0000 UTC m=+34.265008883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:43.192483 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.192462 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:59.19243991 +0000 UTC m=+34.265035586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:43.531180 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:43.531144 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:43.531347 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:43.531294 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:43.531347 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.531300 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:43.531471 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.531413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:43.531471 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:43.531301 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:43.531555 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:43.531543 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:45.527992 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:45.527949 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:45.528421 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:45.527996 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:45.528421 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:45.528080 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:45.528421 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:45.528133 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:45.528421 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:45.528185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:45.528421 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:45.528254 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:47.527263 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.527047 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:47.527761 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.527059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:47.527761 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:47.527388 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:47.527761 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.527100 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:47.527761 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:47.527491 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:47.527761 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:47.527513 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:47.589724 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.589613 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" event={"ID":"143a5776505ae49b336b0210346bca91","Type":"ContainerStarted","Data":"4d17e1a82495ef421794cb37b54d00620f063017fec492a33490434fdf22420a"} Jul 21 18:00:47.591926 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.591899 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f85qv" event={"ID":"964214a8-2f78-4f76-bd48-dc273435e190","Type":"ContainerStarted","Data":"9cdc4c8984dbc102456e14c3f4f46a694f16b6de3eb1c0bb156c0522a7748be0"} Jul 21 18:00:47.594876 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.594857 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:00:47.595610 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.595276 2560 generic.go:356] "Generic (PLEG): container finished" podID="5acea20a-96d0-4ebf-b6be-27366b1575ce" containerID="785a91d1084b15b8c6fabb576b12dbdf32e5a66f76cecafc43cfd42a887a2059" exitCode=1 Jul 21 18:00:47.595610 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.595308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"470806920b8831b71f243b56f7fd3a45f2e37ad3096edfef5f128bd355108916"} Jul 21 18:00:47.595610 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.595384 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"f4b7e94205043785c23c80b946785f7b5269cf5ab4c70cdc9ef0e0f8789ee87f"} Jul 21 18:00:47.595610 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.595438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"d5ae8c679654e7d208116cc8f995875ffede7a9c6e1d72bb5779c060edc8196e"} Jul 21 18:00:47.595610 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.595507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerDied","Data":"785a91d1084b15b8c6fabb576b12dbdf32e5a66f76cecafc43cfd42a887a2059"} Jul 21 18:00:47.595610 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.595558 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"e0070138bcfff71de58dc6ed9e80fbded3a8df3d2dd2df397a70f00c980afb9b"} Jul 21 18:00:47.597181 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.597157 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" event={"ID":"1e01f450-fbc9-45f0-8d02-287ecc687cee","Type":"ContainerStarted","Data":"698ce6a9c4fa3c379d7bf5d640de87c5b47012adf608c7cc5d37a524a3c3850a"} Jul 21 18:00:47.631911 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.631860 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-4.ec2.internal" podStartSLOduration=21.631845501 podStartE2EDuration="21.631845501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:47.631579983 +0000 UTC m=+22.704175682" watchObservedRunningTime="2026-07-21 18:00:47.631845501 +0000 UTC m=+22.704441190" Jul 21 18:00:47.679702 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.679646 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cxnkr" podStartSLOduration=3.536893152 podStartE2EDuration="22.679600893s" totalImagesPullingTime="19.142707741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:47.679559755 +0000 UTC m=+22.752155452" watchObservedRunningTime="2026-07-21 18:00:47.679600893 +0000 UTC m=+22.752196569" Jul 21 18:00:47.728817 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:47.728726 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f85qv" podStartSLOduration=2.528857521 podStartE2EDuration="21.728708276s" totalImagesPullingTime="19.199850755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:47.728477825 +0000 UTC m=+22.801073524" watchObservedRunningTime="2026-07-21 18:00:47.728708276 +0000 UTC m=+22.801303973" Jul 21 18:00:48.601484 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.601236 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fdef49f-195a-4d8f-9e49-6fbb46a08a84" containerID="b8572d0405c0be33a4795ac4c4bd2610c08f3dae071fd31fcfb8e80f64bd7234" exitCode=0 Jul 21 18:00:48.601484 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.601320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerDied","Data":"b8572d0405c0be33a4795ac4c4bd2610c08f3dae071fd31fcfb8e80f64bd7234"} Jul 21 18:00:48.602997 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.602976 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bf96n" event={"ID":"15cb2308-3ce4-450f-a0c9-4c391412109b","Type":"ContainerStarted","Data":"88e89732053dcb60e92b7e023421ecb9cec7f6ec71666e10032a05fd8aeff535"} Jul 21 18:00:48.605802 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.605785 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:00:48.606144 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.606125 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"21b02e0d668a5967e84cc4e6e9a6b7fb0a3ae2f45abcef487ddccae74824b9cc"} Jul 21 18:00:48.607569 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.607542 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" event={"ID":"7481ab0b-595c-4540-8b21-abad89e0aed6","Type":"ContainerStarted","Data":"74c9f0c6a2aed8463d85cb251840c37d54b63570a54214b6ac2ca25f322da04e"} Jul 21 18:00:48.608959 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.608936 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rvrzw" event={"ID":"e85ae40c-e1d2-45cd-87c5-4c63e711ef78","Type":"ContainerStarted","Data":"80116594d75e2cc665e4be317368573bc5ae05122ee82a0ed230618b86143a9a"} Jul 21 18:00:48.610301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.610277 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q64wj" event={"ID":"e07542b9-a38f-4338-8602-60373dc3d93a","Type":"ContainerStarted","Data":"aba5a8e8a6373143692da44b29a80b7d2c8a985a05a808e3939f248034caced0"} Jul 21 18:00:48.611570 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.611546 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8j6h9" event={"ID":"8878bc1b-0748-4045-b076-30860a1b7db4","Type":"ContainerStarted","Data":"a8b32fa3aa3752352d358f58dfbe119328713d1b8e27066dd5ac61a50d6d7a54"} Jul 21 18:00:48.614715 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.614694 2560 generic.go:356] "Generic (PLEG): container finished" podID="fc5664b47cc83a1d98ee9d63320a826b" containerID="251db19385c574e032d57d37a71149d545465a6f0df26ebecd3ce07204c3c1a6" exitCode=0 Jul 21 18:00:48.614825 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.614787 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" event={"ID":"fc5664b47cc83a1d98ee9d63320a826b","Type":"ContainerDied","Data":"251db19385c574e032d57d37a71149d545465a6f0df26ebecd3ce07204c3c1a6"} Jul 21 18:00:48.659294 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.659251 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bf96n" podStartSLOduration=17.686309833 podStartE2EDuration="23.659233502s" totalImagesPullingTime="5.972923669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:48.642155498 +0000 UTC m=+23.714751196" watchObservedRunningTime="2026-07-21 18:00:48.659233502 +0000 UTC m=+23.731829201" Jul 21 18:00:48.673968 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.673926 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8j6h9" podStartSLOduration=4.477672965 podStartE2EDuration="23.673912346s" totalImagesPullingTime="19.196239381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:48.67333986 +0000 UTC m=+23.745935558" watchObservedRunningTime="2026-07-21 18:00:48.673912346 +0000 UTC m=+23.746508044" Jul 21 18:00:48.692628 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.692589 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q64wj" podStartSLOduration=4.533973022 podStartE2EDuration="23.692573222s" totalImagesPullingTime="19.1586002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:48.692128903 +0000 UTC m=+23.764724602" watchObservedRunningTime="2026-07-21 18:00:48.692573222 +0000 UTC m=+23.765168962" Jul 21 18:00:48.713581 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:48.713537 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rvrzw" podStartSLOduration=3.586914018 podStartE2EDuration="22.713524794s" totalImagesPullingTime="19.126610776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:48.713336577 +0000 UTC m=+23.785932274" watchObservedRunningTime="2026-07-21 18:00:48.713524794 +0000 UTC m=+23.786120491" Jul 21 18:00:49.527580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.527543 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:49.527856 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.527587 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:49.527856 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.527545 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:49.527856 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:49.527647 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:49.527856 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:49.527756 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:49.527856 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:49.527831 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:49.618636 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.618604 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" event={"ID":"fc5664b47cc83a1d98ee9d63320a826b","Type":"ContainerStarted","Data":"e7db0bcca45b3584102af2edfd90e85ac61090e609bb0750f27d512f07c69872"} Jul 21 18:00:49.827535 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.827351 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:49.827964 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.827943 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:49.850842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:49.850796 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-4.ec2.internal" podStartSLOduration=23.85077922 podStartE2EDuration="23.85077922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:49.647246759 +0000 UTC m=+24.719842456" watchObservedRunningTime="2026-07-21 18:00:49.85077922 +0000 UTC m=+24.923374917" Jul 21 18:00:50.140670 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.140641 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rvrzw_e85ae40c-e1d2-45cd-87c5-4c63e711ef78/dns-node-resolver/0.log" Jul 21 18:00:50.522975 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.522825 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 18:00:50.624569 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.624541 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:00:50.625063 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.625028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"52a46fc849c68073e268ca4277153108b350a1b8e1a306c90ec4b74e3bd9ee48"} Jul 21 18:00:50.627095 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.627063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" event={"ID":"7481ab0b-595c-4540-8b21-abad89e0aed6","Type":"ContainerStarted","Data":"af5b385957f201951b58f212679abe8b94b4d7231b855eb8e28e7b8a1c06fd66"} Jul 21 18:00:50.627401 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.627370 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:50.627846 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:50.627825 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q64wj" Jul 21 18:00:51.106288 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.106264 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bf96n_15cb2308-3ce4-450f-a0c9-4c391412109b/node-ca/0.log" Jul 21 18:00:51.423200 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.423087 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T18:00:50.522837385Z","UUID":"3d787d9c-48fc-4f0f-9767-fcee6c77becc","Handler":null,"Name":"","Endpoint":""} Jul 21 18:00:51.424712 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.424687 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 18:00:51.424840 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.424728 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 18:00:51.526913 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.526891 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:51.526913 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.526907 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:51.527086 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:51.527005 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:51.527378 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:51.527350 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:51.527511 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.527394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:51.527511 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:51.527465 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:51.631932 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.631897 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" event={"ID":"7481ab0b-595c-4540-8b21-abad89e0aed6","Type":"ContainerStarted","Data":"19c82fbdc766c2d10b26f0a5574d40128896b03efa95e5dd872affcd531c0319"} Jul 21 18:00:51.659145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:51.659089 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j2vc" podStartSLOduration=4.176309372 podStartE2EDuration="26.659069543s" totalImagesPullingTime="22.482760171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:51.658725741 +0000 UTC m=+26.731321439" watchObservedRunningTime="2026-07-21 18:00:51.659069543 +0000 UTC m=+26.731665245" Jul 21 18:00:53.527230 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.527204 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:53.527827 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.527243 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:53.527827 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.527204 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:53.527827 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:53.527327 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:53.527827 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:53.527424 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:53.527827 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:53.527558 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:53.637595 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.637553 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fdef49f-195a-4d8f-9e49-6fbb46a08a84" containerID="d2c3465f36874039c891670975a35fd8f27907a7a920b553d20e42a988761d55" exitCode=0 Jul 21 18:00:53.637732 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.637628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerDied","Data":"d2c3465f36874039c891670975a35fd8f27907a7a920b553d20e42a988761d55"} Jul 21 18:00:53.641642 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.641482 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:00:53.642003 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.641973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"1073cd4852d36c9ae52e775ea8e058cab81f840a8839ddaaae83b27c5736ce59"} Jul 21 18:00:53.642238 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.642222 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:53.642404 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.642385 2560 scope.go:117] "RemoveContainer" containerID="785a91d1084b15b8c6fabb576b12dbdf32e5a66f76cecafc43cfd42a887a2059" Jul 21 18:00:53.660183 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:53.660163 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:54.645926 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.645893 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fdef49f-195a-4d8f-9e49-6fbb46a08a84" containerID="25b8cdd8d643e7a2f700d2c9000b2f3c7b5c513f8e01c0e077755763bc67af43" exitCode=0 Jul 21 18:00:54.646294 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.645979 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerDied","Data":"25b8cdd8d643e7a2f700d2c9000b2f3c7b5c513f8e01c0e077755763bc67af43"} Jul 21 18:00:54.649680 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.649663 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:00:54.650007 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.649987 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" event={"ID":"5acea20a-96d0-4ebf-b6be-27366b1575ce","Type":"ContainerStarted","Data":"4346d56d1b6546052cdc298c3440648c1c7a2dbb527d22e6c9680fb931f8afb1"} Jul 21 18:00:54.650156 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.650144 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:54.650213 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.650182 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:54.665552 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.665530 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:00:54.705568 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:54.705528 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" podStartSLOduration=10.483763243 podStartE2EDuration="29.705515625s" totalImagesPullingTime="19.221752382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:54.70403055 +0000 UTC m=+29.776626251" watchObservedRunningTime="2026-07-21 18:00:54.705515625 +0000 UTC m=+29.778111323" Jul 21 18:00:55.528465 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:55.528363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:55.528465 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:55.528403 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:55.528677 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:55.528556 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:55.528677 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:55.528575 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:55.528677 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:55.528635 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:55.528816 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:55.528675 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:55.658230 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:55.658196 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fdef49f-195a-4d8f-9e49-6fbb46a08a84" containerID="07cf948425479587bf799a634f414dfca505ea7c6e05ddba6091d8d1bb608cea" exitCode=0 Jul 21 18:00:55.658613 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:55.658284 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerDied","Data":"07cf948425479587bf799a634f414dfca505ea7c6e05ddba6091d8d1bb608cea"} Jul 21 18:00:57.526989 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:57.526954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:57.527432 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:57.526995 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:57.527432 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:57.527079 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:57.527432 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:57.527110 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:57.527432 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:57.527167 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:57.527432 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:57.527261 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:59.106523 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.106435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:59.106994 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.106621 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:59.106994 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.106650 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:59.106994 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.106663 2560 projected.go:196] Error preparing data for projected volume kube-api-access-jvbrh for pod openshift-network-diagnostics/network-check-target-2cnsl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:59.106994 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.106734 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh podName:698b7d16-327d-4b00-8ced-1b91bbdd9a56 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:31.106710774 +0000 UTC m=+66.179306474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvbrh" (UniqueName: "kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh") pod "network-check-target-2cnsl" (UID: "698b7d16-327d-4b00-8ced-1b91bbdd9a56") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:59.207412 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.207378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:59.207412 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.207422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:59.207704 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.207540 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:59.207704 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.207542 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:59.207704 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.207601 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret podName:ad0416fc-678d-4d6d-8715-24d76b208eb5 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:31.207584124 +0000 UTC m=+66.280179803 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret") pod "global-pull-secret-syncer-kzvc2" (UID: "ad0416fc-678d-4d6d-8715-24d76b208eb5") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:59.207704 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.207620 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs podName:225d56ab-f30e-44dc-bc64-42071d0e90b8 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:31.207611499 +0000 UTC m=+66.280207180 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs") pod "network-metrics-daemon-ltwt9" (UID: "225d56ab-f30e-44dc-bc64-42071d0e90b8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:59.531204 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.531171 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:00:59.531204 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.531208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:00:59.531571 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.531180 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:00:59.531571 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.531278 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:00:59.531571 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.531392 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:00:59.531571 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:00:59.531521 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:00:59.669745 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:00:59.669713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerStarted","Data":"929de95f2c4b797a1b8c5ab5a21c97df81336b4873fe03d49a9724eb65bc9987"} Jul 21 18:01:00.674862 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:00.674832 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fdef49f-195a-4d8f-9e49-6fbb46a08a84" containerID="929de95f2c4b797a1b8c5ab5a21c97df81336b4873fe03d49a9724eb65bc9987" exitCode=0 Jul 21 18:01:00.675230 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:00.674890 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerDied","Data":"929de95f2c4b797a1b8c5ab5a21c97df81336b4873fe03d49a9724eb65bc9987"} Jul 21 18:01:01.527685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:01.527654 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:01.527685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:01.527674 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:01.527939 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:01.527654 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:01.527939 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:01.527761 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:01.527939 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:01.527852 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:01.527939 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:01.527911 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:01.680237 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:01.680207 2560 generic.go:356] "Generic (PLEG): container finished" podID="0fdef49f-195a-4d8f-9e49-6fbb46a08a84" containerID="3b6252d022336b70a30af3de721757af58f80e807b40210fafce9a29cd8ff7ea" exitCode=0 Jul 21 18:01:01.680631 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:01.680246 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerDied","Data":"3b6252d022336b70a30af3de721757af58f80e807b40210fafce9a29cd8ff7ea"} Jul 21 18:01:02.686143 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:02.686104 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" event={"ID":"0fdef49f-195a-4d8f-9e49-6fbb46a08a84","Type":"ContainerStarted","Data":"bc3a9f393622a5d50904285cf01b3cfa99c0062e06e1f1e55d48febc441ebe73"} Jul 21 18:01:02.715492 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:02.715421 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dcrrq" podStartSLOduration=18.593394177 podStartE2EDuration="37.715404701s" totalImagesPullingTime="19.122010524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:02.714871559 +0000 UTC m=+37.787467257" watchObservedRunningTime="2026-07-21 18:01:02.715404701 +0000 UTC m=+37.788000436" Jul 21 18:01:03.527459 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:03.527416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:03.527626 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:03.527416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:03.527626 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:03.527566 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:03.527720 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:03.527416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:03.527720 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:03.527658 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:03.527792 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:03.527757 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:05.528170 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:05.528131 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:05.528561 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:05.528229 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:05.528561 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:05.528274 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:05.528561 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:05.528270 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:05.528561 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:05.528363 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:05.528561 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:05.528486 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:07.527465 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:07.527410 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:07.527465 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:07.527417 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:07.527849 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:07.527465 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:07.527849 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:07.527562 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:07.527849 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:07.527671 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:07.527849 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:07.527738 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:09.526855 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:09.526816 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:09.526855 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:09.526836 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:09.526855 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:09.526843 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:09.527439 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:09.526953 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:09.527439 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:09.527072 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:09.527439 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:09.527212 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:11.526790 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:11.526753 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:11.527183 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:11.526753 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:11.527183 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:11.526865 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:11.527183 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:11.526920 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:11.527183 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:11.526753 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:11.527183 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:11.526991 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:13.526824 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.526789 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:13.527268 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.526789 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:13.527268 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:13.526929 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:13.527268 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:13.526958 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:13.527268 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.526789 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:13.527268 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:13.527042 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:13.784920 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.784649 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2cnsl"] Jul 21 18:01:13.785058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.785030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:13.785184 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:13.785159 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:13.789518 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.789472 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ltwt9"] Jul 21 18:01:13.789667 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.789629 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:13.789992 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:13.789783 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:13.790564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.790536 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kzvc2"] Jul 21 18:01:13.790682 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:13.790668 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:13.790911 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:13.790837 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:15.528431 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:15.528397 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:15.529017 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:15.528508 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:15.529017 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:15.528586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:15.529017 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:15.528662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:15.529017 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:15.528706 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:15.529017 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:15.528716 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:17.527382 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:17.527337 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:17.527866 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:17.527336 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:17.527866 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:17.527517 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltwt9" podUID="225d56ab-f30e-44dc-bc64-42071d0e90b8" Jul 21 18:01:17.527866 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:17.527352 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:17.527866 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:17.527583 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kzvc2" podUID="ad0416fc-678d-4d6d-8715-24d76b208eb5" Jul 21 18:01:17.527866 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:17.527642 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cnsl" podUID="698b7d16-327d-4b00-8ced-1b91bbdd9a56" Jul 21 18:01:18.263118 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.263090 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-4.ec2.internal" event="NodeReady" Jul 21 18:01:18.263274 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.263208 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 18:01:18.329370 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.329339 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z677t"] Jul 21 18:01:18.351174 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.351132 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2dh7q"] Jul 21 18:01:18.351315 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.351279 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.354116 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.354075 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 18:01:18.354116 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.354107 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 18:01:18.354302 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.354140 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pqxk6\"" Jul 21 18:01:18.366708 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.366690 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z677t"] Jul 21 18:01:18.366804 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.366726 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2dh7q"] Jul 21 18:01:18.366804 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.366797 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.369883 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.369864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 18:01:18.369883 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.369873 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 18:01:18.370014 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.369916 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wbjcp\"" Jul 21 18:01:18.370351 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.370338 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 18:01:18.438295 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.438264 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lsb48"] Jul 21 18:01:18.444809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.444781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e537933-d2d1-4430-bea8-23b23547766a-metrics-tls\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.444918 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.444816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4xtd\" (UniqueName: \"kubernetes.io/projected/623b73a6-41c3-4b45-86ba-f075aeaa1dee-kube-api-access-w4xtd\") pod \"ingress-canary-2dh7q\" (UID: \"623b73a6-41c3-4b45-86ba-f075aeaa1dee\") " pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.444918 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.444836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/623b73a6-41c3-4b45-86ba-f075aeaa1dee-cert\") pod \"ingress-canary-2dh7q\" (UID: \"623b73a6-41c3-4b45-86ba-f075aeaa1dee\") " pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.444918 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.444862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e537933-d2d1-4430-bea8-23b23547766a-tmp-dir\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.445074 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.444943 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vrlz\" (UniqueName: \"kubernetes.io/projected/9e537933-d2d1-4430-bea8-23b23547766a-kube-api-access-8vrlz\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.445074 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.445052 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e537933-d2d1-4430-bea8-23b23547766a-config-volume\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.462928 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.462911 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.468114 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.468094 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lsb48"] Jul 21 18:01:18.473707 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.473688 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 18:01:18.473984 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.473964 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 18:01:18.474067 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.473982 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 18:01:18.474250 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.474235 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 18:01:18.474300 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.474250 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzbvh\"" Jul 21 18:01:18.545872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.545797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4xtd\" (UniqueName: \"kubernetes.io/projected/623b73a6-41c3-4b45-86ba-f075aeaa1dee-kube-api-access-w4xtd\") pod \"ingress-canary-2dh7q\" (UID: \"623b73a6-41c3-4b45-86ba-f075aeaa1dee\") " pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.545872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.545843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/623b73a6-41c3-4b45-86ba-f075aeaa1dee-cert\") pod \"ingress-canary-2dh7q\" (UID: \"623b73a6-41c3-4b45-86ba-f075aeaa1dee\") " pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.545872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.545870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e537933-d2d1-4430-bea8-23b23547766a-tmp-dir\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.545901 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-457gl\" (UniqueName: \"kubernetes.io/projected/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-kube-api-access-457gl\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.545938 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vrlz\" (UniqueName: \"kubernetes.io/projected/9e537933-d2d1-4430-bea8-23b23547766a-kube-api-access-8vrlz\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.545977 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546024 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-crio-socket\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e537933-d2d1-4430-bea8-23b23547766a-config-volume\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546095 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e537933-d2d1-4430-bea8-23b23547766a-metrics-tls\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-data-volume\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.546533 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546266 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e537933-d2d1-4430-bea8-23b23547766a-tmp-dir\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.546810 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.546644 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e537933-d2d1-4430-bea8-23b23547766a-config-volume\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.550052 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.550027 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e537933-d2d1-4430-bea8-23b23547766a-metrics-tls\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.550211 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.550193 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/623b73a6-41c3-4b45-86ba-f075aeaa1dee-cert\") pod \"ingress-canary-2dh7q\" (UID: \"623b73a6-41c3-4b45-86ba-f075aeaa1dee\") " pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.557156 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.557132 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4xtd\" (UniqueName: \"kubernetes.io/projected/623b73a6-41c3-4b45-86ba-f075aeaa1dee-kube-api-access-w4xtd\") pod \"ingress-canary-2dh7q\" (UID: \"623b73a6-41c3-4b45-86ba-f075aeaa1dee\") " pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.557249 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.557190 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vrlz\" (UniqueName: \"kubernetes.io/projected/9e537933-d2d1-4430-bea8-23b23547766a-kube-api-access-8vrlz\") pod \"dns-default-z677t\" (UID: \"9e537933-d2d1-4430-bea8-23b23547766a\") " pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.647463 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.647594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-crio-socket\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.647594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.647594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-data-volume\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.647594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647561 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-457gl\" (UniqueName: \"kubernetes.io/projected/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-kube-api-access-457gl\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.647731 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647593 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-crio-socket\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.647890 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.647870 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-data-volume\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.648077 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.648051 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.649877 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.649860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.658866 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.658845 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-457gl\" (UniqueName: \"kubernetes.io/projected/6bb6b80d-0555-4df3-a29a-bbfe17b96ab0-kube-api-access-457gl\") pod \"insights-runtime-extractor-lsb48\" (UID: \"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0\") " pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.661813 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.661797 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z677t" Jul 21 18:01:18.674924 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.674905 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2dh7q" Jul 21 18:01:18.771588 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.771559 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lsb48" Jul 21 18:01:18.877755 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.877589 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2dh7q"] Jul 21 18:01:18.881207 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:18.881149 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623b73a6_41c3_4b45_86ba_f075aeaa1dee.slice/crio-ad419a682c6fbf7d8ccb029b16e2e0f682313c5a0c92127b949c9abf9976dc69 WatchSource:0}: Error finding container ad419a682c6fbf7d8ccb029b16e2e0f682313c5a0c92127b949c9abf9976dc69: Status 404 returned error can't find the container with id ad419a682c6fbf7d8ccb029b16e2e0f682313c5a0c92127b949c9abf9976dc69 Jul 21 18:01:18.890718 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.890552 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z677t"] Jul 21 18:01:18.892351 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:18.892327 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e537933_d2d1_4430_bea8_23b23547766a.slice/crio-5bb5222c692eb307a16b839b3f69fb299fb092084da9161b0e0774465aa33545 WatchSource:0}: Error finding container 5bb5222c692eb307a16b839b3f69fb299fb092084da9161b0e0774465aa33545: Status 404 returned error can't find the container with id 5bb5222c692eb307a16b839b3f69fb299fb092084da9161b0e0774465aa33545 Jul 21 18:01:18.936093 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:18.936064 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lsb48"] Jul 21 18:01:18.938855 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:18.938830 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb6b80d_0555_4df3_a29a_bbfe17b96ab0.slice/crio-76191f7364e9932bd1de3067af3358cbcdde08a39cc59eba47c44f575389cfa0 WatchSource:0}: Error finding container 76191f7364e9932bd1de3067af3358cbcdde08a39cc59eba47c44f575389cfa0: Status 404 returned error can't find the container with id 76191f7364e9932bd1de3067af3358cbcdde08a39cc59eba47c44f575389cfa0 Jul 21 18:01:19.527333 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.527299 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:19.527728 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.527716 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:19.528044 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.528031 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:19.531609 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.531586 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 18:01:19.531740 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.531610 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 18:01:19.531740 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.531614 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jg5zx\"" Jul 21 18:01:19.531884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.531755 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 18:01:19.531884 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.531851 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 18:01:19.531985 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.531918 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vznt9\"" Jul 21 18:01:19.728073 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.727700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsb48" event={"ID":"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0","Type":"ContainerStarted","Data":"76191f7364e9932bd1de3067af3358cbcdde08a39cc59eba47c44f575389cfa0"} Jul 21 18:01:19.729335 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.729308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z677t" event={"ID":"9e537933-d2d1-4430-bea8-23b23547766a","Type":"ContainerStarted","Data":"5bb5222c692eb307a16b839b3f69fb299fb092084da9161b0e0774465aa33545"} Jul 21 18:01:19.731425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.731400 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2dh7q" event={"ID":"623b73a6-41c3-4b45-86ba-f075aeaa1dee","Type":"ContainerStarted","Data":"ad419a682c6fbf7d8ccb029b16e2e0f682313c5a0c92127b949c9abf9976dc69"} Jul 21 18:01:19.896160 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.896123 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb"] Jul 21 18:01:19.911476 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.911310 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:19.914571 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.914546 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 21 18:01:19.916417 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.916391 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb"] Jul 21 18:01:19.917577 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.917326 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-b7nnt\"" Jul 21 18:01:19.994024 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.993079 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7555b687-j62vv"] Jul 21 18:01:19.997990 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:19.997959 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.003075 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003051 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 18:01:20.003206 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003092 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 18:01:20.003206 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003136 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 18:01:20.003206 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hntdr\"" Jul 21 18:01:20.003557 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 18:01:20.003958 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003439 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 18:01:20.003958 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003943 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 18:01:20.004174 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.003944 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 18:01:20.011433 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.011405 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 18:01:20.015962 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.015936 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7555b687-j62vv"] Jul 21 18:01:20.060756 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-oauth-config\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.060917 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-console-config\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.060917 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060840 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-service-ca\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.060917 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-serving-cert\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.060917 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060905 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q4bx\" (UniqueName: \"kubernetes.io/projected/28838bed-da16-479e-9f52-a880c48de27b-kube-api-access-4q4bx\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.061076 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060934 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-trusted-ca-bundle\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.061076 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060969 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-59jkb\" (UID: \"72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:20.061076 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.060993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-oauth-serving-cert\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.161899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.161816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-oauth-config\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.161899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.161892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-console-config\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.162122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.161918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-service-ca\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.162122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.161956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-serving-cert\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.162122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.161982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q4bx\" (UniqueName: \"kubernetes.io/projected/28838bed-da16-479e-9f52-a880c48de27b-kube-api-access-4q4bx\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.162122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.162008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-trusted-ca-bundle\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.162122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.162041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-59jkb\" (UID: \"72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:20.162122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.162066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-oauth-serving-cert\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.165538 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.163061 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-service-ca\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.165538 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.164697 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-trusted-ca-bundle\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.165538 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:20.165045 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 21 18:01:20.165538 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:20.165100 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1-tls-certificates podName:72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:20.665082271 +0000 UTC m=+55.737677946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-59jkb" (UID: "72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1") : secret "prometheus-operator-admission-webhook-tls" not found Jul 21 18:01:20.166414 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.166365 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-oauth-serving-cert\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.167035 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.166988 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-console-config\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.169233 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.169188 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-oauth-config\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.171790 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.171748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-serving-cert\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.176135 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.176092 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q4bx\" (UniqueName: \"kubernetes.io/projected/28838bed-da16-479e-9f52-a880c48de27b-kube-api-access-4q4bx\") pod \"console-6b7555b687-j62vv\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.310313 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.310001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:20.504030 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.503994 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7555b687-j62vv"] Jul 21 18:01:20.508437 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:20.508395 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28838bed_da16_479e_9f52_a880c48de27b.slice/crio-6265fff41c887939ddc2d8cc40f60d5870377c4be80467b5e4fc6165ff959897 WatchSource:0}: Error finding container 6265fff41c887939ddc2d8cc40f60d5870377c4be80467b5e4fc6165ff959897: Status 404 returned error can't find the container with id 6265fff41c887939ddc2d8cc40f60d5870377c4be80467b5e4fc6165ff959897 Jul 21 18:01:20.666877 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.666837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-59jkb\" (UID: \"72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:20.682415 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.682382 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-59jkb\" (UID: \"72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:20.738218 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.738124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7555b687-j62vv" event={"ID":"28838bed-da16-479e-9f52-a880c48de27b","Type":"ContainerStarted","Data":"6265fff41c887939ddc2d8cc40f60d5870377c4be80467b5e4fc6165ff959897"} Jul 21 18:01:20.824273 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:20.824224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:26.367389 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.367359 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb"] Jul 21 18:01:26.372714 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:26.372676 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ebc1e3_2eb4_4b9a_a099_4757a8f71ef1.slice/crio-91bd4e566644a829e08c1c4f0c6b63c5bc88940f9db26b68259661e45ebe8eee WatchSource:0}: Error finding container 91bd4e566644a829e08c1c4f0c6b63c5bc88940f9db26b68259661e45ebe8eee: Status 404 returned error can't find the container with id 91bd4e566644a829e08c1c4f0c6b63c5bc88940f9db26b68259661e45ebe8eee Jul 21 18:01:26.672137 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.672108 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzqwp" Jul 21 18:01:26.757570 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.757529 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" event={"ID":"72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1","Type":"ContainerStarted","Data":"91bd4e566644a829e08c1c4f0c6b63c5bc88940f9db26b68259661e45ebe8eee"} Jul 21 18:01:26.759257 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.759232 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z677t" event={"ID":"9e537933-d2d1-4430-bea8-23b23547766a","Type":"ContainerStarted","Data":"af247d922bae1448fdc861458cdeecb93950a07e288e2875b9f17de894150c55"} Jul 21 18:01:26.759388 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.759281 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z677t" event={"ID":"9e537933-d2d1-4430-bea8-23b23547766a","Type":"ContainerStarted","Data":"4001239c36e8de0ee514afdd570ba65f984fd49ce37cdefcc313770517561446"} Jul 21 18:01:26.759478 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.759458 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z677t" Jul 21 18:01:26.760821 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.760792 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7555b687-j62vv" event={"ID":"28838bed-da16-479e-9f52-a880c48de27b","Type":"ContainerStarted","Data":"a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e"} Jul 21 18:01:26.762279 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.762258 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2dh7q" event={"ID":"623b73a6-41c3-4b45-86ba-f075aeaa1dee","Type":"ContainerStarted","Data":"e10d7adc514f9d227cf604981cb93206dedb75d7f2231b9a2745354723f8d62d"} Jul 21 18:01:26.763654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.763634 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsb48" event={"ID":"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0","Type":"ContainerStarted","Data":"7c32685c79f84b75b9185b04c4db4239a970f692ff0496197a6115efecd64b87"} Jul 21 18:01:26.826584 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.824113 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z677t" podStartSLOduration=1.832329382 podStartE2EDuration="8.824087088s" totalImagesPullingTime="6.991757706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:26.795418347 +0000 UTC m=+61.868014046" watchObservedRunningTime="2026-07-21 18:01:26.824087088 +0000 UTC m=+61.896682788" Jul 21 18:01:26.826584 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.824261 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2dh7q" podStartSLOduration=1.492068832 podStartE2EDuration="8.824253203s" totalImagesPullingTime="7.332184371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:26.823177929 +0000 UTC m=+61.895773638" watchObservedRunningTime="2026-07-21 18:01:26.824253203 +0000 UTC m=+61.896848901" Jul 21 18:01:26.860553 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:26.860490 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7555b687-j62vv" podStartSLOduration=2.15658074 podStartE2EDuration="7.860466371s" totalImagesPullingTime="5.703885631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:26.85924535 +0000 UTC m=+61.931841051" watchObservedRunningTime="2026-07-21 18:01:26.860466371 +0000 UTC m=+61.933062061" Jul 21 18:01:27.768307 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.768269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" event={"ID":"72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1","Type":"ContainerStarted","Data":"f9d7ec57b1050af9aa1200b68e1403173fc07f4fe669fd913b11ec0de0eb2e0b"} Jul 21 18:01:27.768794 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.768781 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:27.774191 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.774170 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" Jul 21 18:01:27.786580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.786532 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-59jkb" podStartSLOduration=7.54145042 podStartE2EDuration="8.786518945s" totalImagesPullingTime="1.245068525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:27.784978623 +0000 UTC m=+62.857574323" watchObservedRunningTime="2026-07-21 18:01:27.786518945 +0000 UTC m=+62.859114637" Jul 21 18:01:27.988383 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.988303 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw"] Jul 21 18:01:27.992464 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.992429 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:27.996286 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.996253 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 18:01:27.996500 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.996288 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 18:01:27.996500 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.996382 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 21 18:01:27.996500 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.996410 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zp85k\"" Jul 21 18:01:27.996974 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.996953 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 21 18:01:27.997082 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:27.996969 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 18:01:28.006414 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.006388 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw"] Jul 21 18:01:28.025481 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.025434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5eca6724-e762-40b4-ac32-e6805163e231-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.025481 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.025482 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eca6724-e762-40b4-ac32-e6805163e231-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.025692 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.025501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlzjm\" (UniqueName: \"kubernetes.io/projected/5eca6724-e762-40b4-ac32-e6805163e231-kube-api-access-xlzjm\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.025692 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.025604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5eca6724-e762-40b4-ac32-e6805163e231-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.126127 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.126085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlzjm\" (UniqueName: \"kubernetes.io/projected/5eca6724-e762-40b4-ac32-e6805163e231-kube-api-access-xlzjm\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.126315 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.126143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5eca6724-e762-40b4-ac32-e6805163e231-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.126315 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.126244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5eca6724-e762-40b4-ac32-e6805163e231-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.126315 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.126274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eca6724-e762-40b4-ac32-e6805163e231-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.127087 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.126976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eca6724-e762-40b4-ac32-e6805163e231-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.129324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.129301 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5eca6724-e762-40b4-ac32-e6805163e231-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.129429 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.129384 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5eca6724-e762-40b4-ac32-e6805163e231-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.135582 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.135560 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlzjm\" (UniqueName: \"kubernetes.io/projected/5eca6724-e762-40b4-ac32-e6805163e231-kube-api-access-xlzjm\") pod \"prometheus-operator-5945b5cfd-qb8vw\" (UID: \"5eca6724-e762-40b4-ac32-e6805163e231\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:28.303555 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:28.303474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" Jul 21 18:01:29.887406 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:29.887373 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw"] Jul 21 18:01:29.892654 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:29.892630 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eca6724_e762_40b4_ac32_e6805163e231.slice/crio-27667bc839b61b54d37c2659affd1a1ee9521117af64c9ac1c6fa4e76187bce9 WatchSource:0}: Error finding container 27667bc839b61b54d37c2659affd1a1ee9521117af64c9ac1c6fa4e76187bce9: Status 404 returned error can't find the container with id 27667bc839b61b54d37c2659affd1a1ee9521117af64c9ac1c6fa4e76187bce9 Jul 21 18:01:30.310380 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:30.310346 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:30.310563 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:30.310403 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:30.314991 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:30.314968 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:30.780607 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:30.780564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsb48" event={"ID":"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0","Type":"ContainerStarted","Data":"e373f2ab235e01eeb656da1cd8aa5d35b5ca2bbf297645809d21a2d19a4aa02c"} Jul 21 18:01:30.781798 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:30.781768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" event={"ID":"5eca6724-e762-40b4-ac32-e6805163e231","Type":"ContainerStarted","Data":"27667bc839b61b54d37c2659affd1a1ee9521117af64c9ac1c6fa4e76187bce9"} Jul 21 18:01:30.786307 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:30.786285 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:01:31.152603 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.152570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:31.157293 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.157265 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 18:01:31.166489 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.166437 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 18:01:31.177499 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.177469 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvbrh\" (UniqueName: \"kubernetes.io/projected/698b7d16-327d-4b00-8ced-1b91bbdd9a56-kube-api-access-jvbrh\") pod \"network-check-target-2cnsl\" (UID: \"698b7d16-327d-4b00-8ced-1b91bbdd9a56\") " pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:31.253209 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.253162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:31.253392 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.253219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:31.258393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.258361 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 18:01:31.258393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.258393 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 18:01:31.266215 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.266100 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/225d56ab-f30e-44dc-bc64-42071d0e90b8-metrics-certs\") pod \"network-metrics-daemon-ltwt9\" (UID: \"225d56ab-f30e-44dc-bc64-42071d0e90b8\") " pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:31.266215 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.266181 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad0416fc-678d-4d6d-8715-24d76b208eb5-original-pull-secret\") pod \"global-pull-secret-syncer-kzvc2\" (UID: \"ad0416fc-678d-4d6d-8715-24d76b208eb5\") " pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:31.285395 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.285363 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jg5zx\"" Jul 21 18:01:31.289978 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.289954 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vznt9\"" Jul 21 18:01:31.291010 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.290978 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltwt9" Jul 21 18:01:31.297550 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.297513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:31.471134 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.471052 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2cnsl"] Jul 21 18:01:31.478667 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:31.478570 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698b7d16_327d_4b00_8ced_1b91bbdd9a56.slice/crio-a22ab43b112e60e83d98d4017b397a87b30cc86aebd2a327c29c291db507af38 WatchSource:0}: Error finding container a22ab43b112e60e83d98d4017b397a87b30cc86aebd2a327c29c291db507af38: Status 404 returned error can't find the container with id a22ab43b112e60e83d98d4017b397a87b30cc86aebd2a327c29c291db507af38 Jul 21 18:01:31.495083 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.494501 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ltwt9"] Jul 21 18:01:31.498925 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:31.498895 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod225d56ab_f30e_44dc_bc64_42071d0e90b8.slice/crio-4df2218ab5b81bfa6b12765599625e34f18ea11d292031c6f9b2d7782b4cda76 WatchSource:0}: Error finding container 4df2218ab5b81bfa6b12765599625e34f18ea11d292031c6f9b2d7782b4cda76: Status 404 returned error can't find the container with id 4df2218ab5b81bfa6b12765599625e34f18ea11d292031c6f9b2d7782b4cda76 Jul 21 18:01:31.544652 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.544622 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kzvc2" Jul 21 18:01:31.709932 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.709848 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kzvc2"] Jul 21 18:01:31.713624 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:31.713591 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0416fc_678d_4d6d_8715_24d76b208eb5.slice/crio-979bc5f86d2ee6cfcf99910bb09ef1274016e5666624fb3b278196dadc5add30 WatchSource:0}: Error finding container 979bc5f86d2ee6cfcf99910bb09ef1274016e5666624fb3b278196dadc5add30: Status 404 returned error can't find the container with id 979bc5f86d2ee6cfcf99910bb09ef1274016e5666624fb3b278196dadc5add30 Jul 21 18:01:31.787362 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.787328 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kzvc2" event={"ID":"ad0416fc-678d-4d6d-8715-24d76b208eb5","Type":"ContainerStarted","Data":"979bc5f86d2ee6cfcf99910bb09ef1274016e5666624fb3b278196dadc5add30"} Jul 21 18:01:31.788886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.788856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2cnsl" event={"ID":"698b7d16-327d-4b00-8ced-1b91bbdd9a56","Type":"ContainerStarted","Data":"a22ab43b112e60e83d98d4017b397a87b30cc86aebd2a327c29c291db507af38"} Jul 21 18:01:31.790941 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.790902 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" event={"ID":"5eca6724-e762-40b4-ac32-e6805163e231","Type":"ContainerStarted","Data":"d1544062b7549bf2d38bd98ea1e81de136e4a91457746bb2ef1e2ca0c1d2dbd7"} Jul 21 18:01:31.791099 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.790969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" event={"ID":"5eca6724-e762-40b4-ac32-e6805163e231","Type":"ContainerStarted","Data":"19bdfeffb978a1929fdd3d2cb6259d777b3282a5a6958779f4ce20f057452151"} Jul 21 18:01:31.792316 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.792287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ltwt9" event={"ID":"225d56ab-f30e-44dc-bc64-42071d0e90b8","Type":"ContainerStarted","Data":"4df2218ab5b81bfa6b12765599625e34f18ea11d292031c6f9b2d7782b4cda76"} Jul 21 18:01:31.813165 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:31.813121 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-qb8vw" podStartSLOduration=3.358863566 podStartE2EDuration="4.813109087s" totalImagesPullingTime="1.454245521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:31.812556392 +0000 UTC m=+66.885152092" watchObservedRunningTime="2026-07-21 18:01:31.813109087 +0000 UTC m=+66.885704784" Jul 21 18:01:33.389554 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.389513 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7"] Jul 21 18:01:33.405674 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.405638 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-rzn86"] Jul 21 18:01:33.407345 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.407319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.411608 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.411263 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 18:01:33.411608 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.411403 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 18:01:33.411813 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.411663 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hfjbs\"" Jul 21 18:01:33.427842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.427808 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7"] Jul 21 18:01:33.427996 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.427865 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-rzn86"] Jul 21 18:01:33.427996 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.427899 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sz72g"] Jul 21 18:01:33.429510 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.429410 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.433165 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.433142 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 21 18:01:33.433397 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.433379 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 21 18:01:33.433755 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.433735 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 21 18:01:33.433958 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.433941 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pgdlg\"" Jul 21 18:01:33.447650 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.447622 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.451344 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.451166 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 18:01:33.451344 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.451245 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q5ldh\"" Jul 21 18:01:33.451572 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.451532 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 18:01:33.451679 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.451658 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-wtmp\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrjk4\" (UniqueName: \"kubernetes.io/projected/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-kube-api-access-vrjk4\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-root\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45beceea-a768-4f04-a69a-e54e0431fea6-metrics-client-ca\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee88fd76-5152-4601-a70a-b041335879d1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-textfile\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472860 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z44jk\" (UniqueName: \"kubernetes.io/projected/ee88fd76-5152-4601-a70a-b041335879d1-kube-api-access-z44jk\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-accelerators-collector-config\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472909 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-tls\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473182 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472930 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ee88fd76-5152-4601-a70a-b041335879d1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.473905 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.473905 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472967 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-sys\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473905 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.472990 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.473905 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.473021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klpq5\" (UniqueName: \"kubernetes.io/projected/45beceea-a768-4f04-a69a-e54e0431fea6-kube-api-access-klpq5\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.573634 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.573634 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-wtmp\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573661 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrjk4\" (UniqueName: \"kubernetes.io/projected/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-kube-api-access-vrjk4\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573746 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573767 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-root\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573780 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-wtmp\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45beceea-a768-4f04-a69a-e54e0431fea6-metrics-client-ca\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee88fd76-5152-4601-a70a-b041335879d1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.573886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-textfile\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z44jk\" (UniqueName: \"kubernetes.io/projected/ee88fd76-5152-4601-a70a-b041335879d1-kube-api-access-z44jk\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-accelerators-collector-config\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-tls\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.573995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ee88fd76-5152-4601-a70a-b041335879d1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.574012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.574041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-sys\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.574082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.574120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klpq5\" (UniqueName: \"kubernetes.io/projected/45beceea-a768-4f04-a69a-e54e0431fea6-kube-api-access-klpq5\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.574424 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.574359 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45beceea-a768-4f04-a69a-e54e0431fea6-metrics-client-ca\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.575089 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.574848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.575089 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.575058 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee88fd76-5152-4601-a70a-b041335879d1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.575193 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.575138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-root\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.575340 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.575316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-textfile\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.575499 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:33.575404 2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 18:01:33.575499 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.575414 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ee88fd76-5152-4601-a70a-b041335879d1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.576070 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:33.575876 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 18:01:33.576070 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.575891 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-accelerators-collector-config\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.576070 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:33.575947 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-tls podName:45beceea-a768-4f04-a69a-e54e0431fea6 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:34.075927751 +0000 UTC m=+69.148523426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-tls") pod "node-exporter-sz72g" (UID: "45beceea-a768-4f04-a69a-e54e0431fea6") : secret "node-exporter-tls" not found Jul 21 18:01:33.576070 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.576007 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.576551 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.576165 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45beceea-a768-4f04-a69a-e54e0431fea6-sys\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.576690 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:33.576660 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-tls podName:80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:34.076639026 +0000 UTC m=+69.149234715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-zznq7" (UID: "80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181") : secret "openshift-state-metrics-tls" not found Jul 21 18:01:33.577257 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.577232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.577685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.577664 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.579004 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.578974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee88fd76-5152-4601-a70a-b041335879d1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.583611 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.583559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrjk4\" (UniqueName: \"kubernetes.io/projected/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-kube-api-access-vrjk4\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:33.583611 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.583594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klpq5\" (UniqueName: \"kubernetes.io/projected/45beceea-a768-4f04-a69a-e54e0431fea6-kube-api-access-klpq5\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.584057 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.584036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:33.587200 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.587143 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z44jk\" (UniqueName: \"kubernetes.io/projected/ee88fd76-5152-4601-a70a-b041335879d1-kube-api-access-z44jk\") pod \"kube-state-metrics-56c5594674-rzn86\" (UID: \"ee88fd76-5152-4601-a70a-b041335879d1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:33.742931 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:33.742836 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" Jul 21 18:01:34.078566 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.078477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:34.078735 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.078564 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-tls\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:34.081231 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.081201 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45beceea-a768-4f04-a69a-e54e0431fea6-node-exporter-tls\") pod \"node-exporter-sz72g\" (UID: \"45beceea-a768-4f04-a69a-e54e0431fea6\") " pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:34.081527 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.081508 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zznq7\" (UID: \"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:34.322394 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.322355 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" Jul 21 18:01:34.359509 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.359473 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sz72g" Jul 21 18:01:34.632712 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.632671 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 18:01:34.635703 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.635670 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.638970 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.638946 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 18:01:34.638970 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.638963 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 18:01:34.639202 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.639016 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t7d2z\"" Jul 21 18:01:34.639202 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.638968 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 18:01:34.639897 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.639878 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 18:01:34.640002 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.639970 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 18:01:34.640060 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.640045 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 18:01:34.640124 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.640076 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 18:01:34.640377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.640358 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 18:01:34.640469 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.640430 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 18:01:34.667251 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.667222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 18:01:34.682674 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.682842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtffm\" (UniqueName: \"kubernetes.io/projected/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-kube-api-access-jtffm\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.682842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.682842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682736 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.682842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.682842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-config-out\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-web-config\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.682998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683172 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.683038 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.683172 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.683109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.783566 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.783566 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.783798 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:34.783674 2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 21 18:01:34.783798 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtffm\" (UniqueName: \"kubernetes.io/projected/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-kube-api-access-jtffm\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.783798 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.783798 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:01:34.783746 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-main-tls podName:f1448cc9-0af4-47aa-86f3-9e13c8a45f25 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:35.283729789 +0000 UTC m=+70.356325465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f1448cc9-0af4-47aa-86f3-9e13c8a45f25") : secret "alertmanager-main-tls" not found Jul 21 18:01:34.783798 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784039 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783809 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784039 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784039 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.783972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784039 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-config-out\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784248 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784063 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-web-config\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784248 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784086 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784248 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784248 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784545 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784528 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.784725 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.784702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.785660 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.785633 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.787190 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.787162 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.787325 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.787298 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.787464 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.787424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.787833 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.787787 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-web-config\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.788343 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.788311 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.788606 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.788582 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-config-out\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.788714 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.788690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.789267 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.789250 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:34.797472 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:34.797429 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtffm\" (UniqueName: \"kubernetes.io/projected/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-kube-api-access-jtffm\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:35.289265 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.289157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:35.292766 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.292726 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1448cc9-0af4-47aa-86f3-9e13c8a45f25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1448cc9-0af4-47aa-86f3-9e13c8a45f25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:35.546297 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.546218 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 18:01:35.692386 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:35.692346 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45beceea_a768_4f04_a69a_e54e0431fea6.slice/crio-388ff6efe69de5ff7355c59daf67d597e1af9b2db9a83c303236982e1ab37467 WatchSource:0}: Error finding container 388ff6efe69de5ff7355c59daf67d597e1af9b2db9a83c303236982e1ab37467: Status 404 returned error can't find the container with id 388ff6efe69de5ff7355c59daf67d597e1af9b2db9a83c303236982e1ab37467 Jul 21 18:01:35.818191 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.817971 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2cnsl" event={"ID":"698b7d16-327d-4b00-8ced-1b91bbdd9a56","Type":"ContainerStarted","Data":"4c053c4d5dcd548ffb0906973eee612412778d86a10b5a6d5513291a8455f846"} Jul 21 18:01:35.818191 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.818051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:01:35.821418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.821354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsb48" event={"ID":"6bb6b80d-0555-4df3-a29a-bbfe17b96ab0","Type":"ContainerStarted","Data":"50cbaf3b7a6a2cbab74e916fd2adba8c68cb23d1a1e3d42f22e977691b182361"} Jul 21 18:01:35.822969 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.822941 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sz72g" event={"ID":"45beceea-a768-4f04-a69a-e54e0431fea6","Type":"ContainerStarted","Data":"388ff6efe69de5ff7355c59daf67d597e1af9b2db9a83c303236982e1ab37467"} Jul 21 18:01:35.842482 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.842293 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2cnsl" podStartSLOduration=66.618516856 podStartE2EDuration="1m10.842274692s" totalImagesPullingTime="4.223757836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:35.841780103 +0000 UTC m=+70.914375801" watchObservedRunningTime="2026-07-21 18:01:35.842274692 +0000 UTC m=+70.914870390" Jul 21 18:01:35.869685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.868357 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lsb48" podStartSLOduration=1.278200321 podStartE2EDuration="17.868326426s" totalImagesPullingTime="16.590126105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:01:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:35.866424808 +0000 UTC m=+70.939020530" watchObservedRunningTime="2026-07-21 18:01:35.868326426 +0000 UTC m=+70.940922125" Jul 21 18:01:35.916893 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.916866 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 18:01:35.921766 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:35.921683 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1448cc9_0af4_47aa_86f3_9e13c8a45f25.slice/crio-2579cf78a33bd30cbf7b0a85860b37d8046a42a9dae8520894b140fc71608a68 WatchSource:0}: Error finding container 2579cf78a33bd30cbf7b0a85860b37d8046a42a9dae8520894b140fc71608a68: Status 404 returned error can't find the container with id 2579cf78a33bd30cbf7b0a85860b37d8046a42a9dae8520894b140fc71608a68 Jul 21 18:01:35.946542 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.946519 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7"] Jul 21 18:01:35.948028 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:35.948002 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-rzn86"] Jul 21 18:01:35.950786 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:35.950754 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d63e9c_bbcb_4dce_b8e5_eb8b5adf4181.slice/crio-004eb8d8cdf922d16a73cb7f822faec4a7203e1567ca0f9f0d47d1fb7aef6066 WatchSource:0}: Error finding container 004eb8d8cdf922d16a73cb7f822faec4a7203e1567ca0f9f0d47d1fb7aef6066: Status 404 returned error can't find the container with id 004eb8d8cdf922d16a73cb7f822faec4a7203e1567ca0f9f0d47d1fb7aef6066 Jul 21 18:01:35.958904 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:35.958880 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee88fd76_5152_4601_a70a_b041335879d1.slice/crio-4d09ecd9c9c182cbe4a551795a4cff73aaa0b52fd44b21ba4a7d5a6a3eed214d WatchSource:0}: Error finding container 4d09ecd9c9c182cbe4a551795a4cff73aaa0b52fd44b21ba4a7d5a6a3eed214d: Status 404 returned error can't find the container with id 4d09ecd9c9c182cbe4a551795a4cff73aaa0b52fd44b21ba4a7d5a6a3eed214d Jul 21 18:01:36.596117 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.590881 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78b4658575-flbgw"] Jul 21 18:01:36.602597 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.601826 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.607580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.607556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 21 18:01:36.607841 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.607818 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f2706js0ahqu9\"" Jul 21 18:01:36.608305 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.608177 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ghmq9\"" Jul 21 18:01:36.608305 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.608197 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 21 18:01:36.608796 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.608563 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 21 18:01:36.609589 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.609574 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 21 18:01:36.609941 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.607615 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 21 18:01:36.610080 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.610063 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78b4658575-flbgw"] Jul 21 18:01:36.700774 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700736 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700831 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9536aa4a-a08d-47dc-a51e-b3128755c993-metrics-client-ca\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdsp\" (UniqueName: \"kubernetes.io/projected/9536aa4a-a08d-47dc-a51e-b3128755c993-kube-api-access-zbdsp\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700893 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700909 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-grpc-tls\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.701240 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.700936 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-tls\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.771598 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.770718 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z677t" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.801870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.801937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.801972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.802001 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9536aa4a-a08d-47dc-a51e-b3128755c993-metrics-client-ca\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.802033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdsp\" (UniqueName: \"kubernetes.io/projected/9536aa4a-a08d-47dc-a51e-b3128755c993-kube-api-access-zbdsp\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.802070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.802097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-grpc-tls\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.802280 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.802143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-tls\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.803371 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.803314 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9536aa4a-a08d-47dc-a51e-b3128755c993-metrics-client-ca\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.806020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.805999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.814232 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.814183 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.817941 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.817070 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-grpc-tls\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.818734 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.818708 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.820004 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.819978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.820494 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.820425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9536aa4a-a08d-47dc-a51e-b3128755c993-secret-thanos-querier-tls\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.831746 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.831526 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ltwt9" event={"ID":"225d56ab-f30e-44dc-bc64-42071d0e90b8","Type":"ContainerStarted","Data":"1711b68e85fa0296f56be219e2f086196280354e50a655652d4c1f0e8861893a"} Jul 21 18:01:36.831746 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.831585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ltwt9" event={"ID":"225d56ab-f30e-44dc-bc64-42071d0e90b8","Type":"ContainerStarted","Data":"ca203f8ae558dac458f143e38817953d687665197ac6abba9110aed5591ee3b5"} Jul 21 18:01:36.834977 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.834826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" event={"ID":"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181","Type":"ContainerStarted","Data":"3e1e96e098868d84bf2dd0bff3fac7234f04798785b5170cb372e3ff9768d26b"} Jul 21 18:01:36.834977 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.834891 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" event={"ID":"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181","Type":"ContainerStarted","Data":"fe801149ab330aff7e745a7b295fbe093736b27ab206ec2da9a299c8232b361e"} Jul 21 18:01:36.834977 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.834923 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" event={"ID":"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181","Type":"ContainerStarted","Data":"004eb8d8cdf922d16a73cb7f822faec4a7203e1567ca0f9f0d47d1fb7aef6066"} Jul 21 18:01:36.835458 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.835386 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdsp\" (UniqueName: \"kubernetes.io/projected/9536aa4a-a08d-47dc-a51e-b3128755c993-kube-api-access-zbdsp\") pod \"thanos-querier-78b4658575-flbgw\" (UID: \"9536aa4a-a08d-47dc-a51e-b3128755c993\") " pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:36.837937 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.837861 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" event={"ID":"ee88fd76-5152-4601-a70a-b041335879d1","Type":"ContainerStarted","Data":"4d09ecd9c9c182cbe4a551795a4cff73aaa0b52fd44b21ba4a7d5a6a3eed214d"} Jul 21 18:01:36.840509 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.839788 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"2579cf78a33bd30cbf7b0a85860b37d8046a42a9dae8520894b140fc71608a68"} Jul 21 18:01:36.853277 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.853190 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ltwt9" podStartSLOduration=67.661650941 podStartE2EDuration="1m11.853171181s" totalImagesPullingTime="4.19152024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:36.851657012 +0000 UTC m=+71.924252711" watchObservedRunningTime="2026-07-21 18:01:36.853171181 +0000 UTC m=+71.925766879" Jul 21 18:01:36.914806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:36.914509 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:37.198434 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.198386 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78b4658575-flbgw"] Jul 21 18:01:37.793745 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.793709 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c5b88859d-g2wgs"] Jul 21 18:01:37.797261 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.797227 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.800683 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.800657 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 21 18:01:37.801061 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.800851 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 21 18:01:37.801526 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.801504 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-trg44\"" Jul 21 18:01:37.802418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.801702 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 21 18:01:37.802418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.801792 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eqmfncj62o1bg\"" Jul 21 18:01:37.802418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.801933 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 21 18:01:37.811502 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.811472 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c5b88859d-g2wgs"] Jul 21 18:01:37.914761 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.914709 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-secret-metrics-server-client-certs\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.914761 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.914762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d2069994-290e-4e72-a12f-8e509e333ca7-metrics-server-audit-profiles\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.915007 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.914882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np4wc\" (UniqueName: \"kubernetes.io/projected/d2069994-290e-4e72-a12f-8e509e333ca7-kube-api-access-np4wc\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.915007 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.914952 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-secret-metrics-server-tls\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.915109 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.915021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d2069994-290e-4e72-a12f-8e509e333ca7-audit-log\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.915109 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.915056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-client-ca-bundle\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:37.915109 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:37.915091 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2069994-290e-4e72-a12f-8e509e333ca7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np4wc\" (UniqueName: \"kubernetes.io/projected/d2069994-290e-4e72-a12f-8e509e333ca7-kube-api-access-np4wc\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-secret-metrics-server-tls\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d2069994-290e-4e72-a12f-8e509e333ca7-audit-log\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016149 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-client-ca-bundle\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2069994-290e-4e72-a12f-8e509e333ca7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-secret-metrics-server-client-certs\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.016249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d2069994-290e-4e72-a12f-8e509e333ca7-metrics-server-audit-profiles\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.017809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.017355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d2069994-290e-4e72-a12f-8e509e333ca7-metrics-server-audit-profiles\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.019132 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.018674 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d2069994-290e-4e72-a12f-8e509e333ca7-audit-log\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.019396 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.019356 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2069994-290e-4e72-a12f-8e509e333ca7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.020758 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.020732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-client-ca-bundle\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.021228 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.021185 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-secret-metrics-server-tls\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.021960 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.021940 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d2069994-290e-4e72-a12f-8e509e333ca7-secret-metrics-server-client-certs\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.031348 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.031327 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np4wc\" (UniqueName: \"kubernetes.io/projected/d2069994-290e-4e72-a12f-8e509e333ca7-kube-api-access-np4wc\") pod \"metrics-server-6c5b88859d-g2wgs\" (UID: \"d2069994-290e-4e72-a12f-8e509e333ca7\") " pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.114140 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.114064 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:38.163058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.161732 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-555dc88856-fbbxs"] Jul 21 18:01:38.165144 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.165121 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.174865 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.174843 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc"] Jul 21 18:01:38.178215 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.178192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.185689 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.185637 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zcq95\"" Jul 21 18:01:38.186034 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.186015 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 21 18:01:38.189379 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.189351 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555dc88856-fbbxs"] Jul 21 18:01:38.200073 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.200045 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc"] Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.217907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkh4n\" (UniqueName: \"kubernetes.io/projected/ae64415c-2f98-4d23-854d-adc8fef068c9-kube-api-access-kkh4n\") pod \"monitoring-plugin-7b7b7bc8bf-qnmtc\" (UID: \"ae64415c-2f98-4d23-854d-adc8fef068c9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.217995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-serving-cert\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218024 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-trusted-ca-bundle\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218053 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae64415c-2f98-4d23-854d-adc8fef068c9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qnmtc\" (UID: \"ae64415c-2f98-4d23-854d-adc8fef068c9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218079 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98mnh\" (UniqueName: \"kubernetes.io/projected/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-kube-api-access-98mnh\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218133 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-service-ca\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218169 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-oauth-config\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218225 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-config\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.218377 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.218252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-oauth-serving-cert\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.314730 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:38.314698 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9536aa4a_a08d_47dc_a51e_b3128755c993.slice/crio-b2d26fb4c6acbff3ce852fb25165c5c614e992f31a8a836591ecdccacf76d893 WatchSource:0}: Error finding container b2d26fb4c6acbff3ce852fb25165c5c614e992f31a8a836591ecdccacf76d893: Status 404 returned error can't find the container with id b2d26fb4c6acbff3ce852fb25165c5c614e992f31a8a836591ecdccacf76d893 Jul 21 18:01:38.319209 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-serving-cert\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-trusted-ca-bundle\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae64415c-2f98-4d23-854d-adc8fef068c9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qnmtc\" (UID: \"ae64415c-2f98-4d23-854d-adc8fef068c9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.319314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98mnh\" (UniqueName: \"kubernetes.io/projected/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-kube-api-access-98mnh\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-service-ca\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319539 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-oauth-config\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319539 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-config\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319753 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-oauth-serving-cert\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.319856 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.319778 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkh4n\" (UniqueName: \"kubernetes.io/projected/ae64415c-2f98-4d23-854d-adc8fef068c9-kube-api-access-kkh4n\") pod \"monitoring-plugin-7b7b7bc8bf-qnmtc\" (UID: \"ae64415c-2f98-4d23-854d-adc8fef068c9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.320068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.320045 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-service-ca\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.320270 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.320247 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-config\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.320523 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.320496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-trusted-ca-bundle\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.320591 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.320559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-oauth-serving-cert\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.321974 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.321949 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-serving-cert\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.322068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.322037 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae64415c-2f98-4d23-854d-adc8fef068c9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qnmtc\" (UID: \"ae64415c-2f98-4d23-854d-adc8fef068c9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.322139 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.322122 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-oauth-config\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.330580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.330559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98mnh\" (UniqueName: \"kubernetes.io/projected/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-kube-api-access-98mnh\") pod \"console-555dc88856-fbbxs\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.331113 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.331097 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkh4n\" (UniqueName: \"kubernetes.io/projected/ae64415c-2f98-4d23-854d-adc8fef068c9-kube-api-access-kkh4n\") pod \"monitoring-plugin-7b7b7bc8bf-qnmtc\" (UID: \"ae64415c-2f98-4d23-854d-adc8fef068c9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.482234 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.479964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:38.490197 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.490168 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:38.617950 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.615139 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c5b88859d-g2wgs"] Jul 21 18:01:38.623052 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:38.623017 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2069994_290e_4e72_a12f_8e509e333ca7.slice/crio-c280ee76f5333fb608af9f22550b00e2a237ab639743e17dc33ff5920712a8df WatchSource:0}: Error finding container c280ee76f5333fb608af9f22550b00e2a237ab639743e17dc33ff5920712a8df: Status 404 returned error can't find the container with id c280ee76f5333fb608af9f22550b00e2a237ab639743e17dc33ff5920712a8df Jul 21 18:01:38.713862 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.713760 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-555dc88856-fbbxs"] Jul 21 18:01:38.731620 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.731590 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc"] Jul 21 18:01:38.732991 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:38.732948 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e51ba0f_1598_4e5c_b9ac_3e028b475e61.slice/crio-80251af015d263cb4babcda9bdba2af729e5e005cafd0813bb616aa2ff925eca WatchSource:0}: Error finding container 80251af015d263cb4babcda9bdba2af729e5e005cafd0813bb616aa2ff925eca: Status 404 returned error can't find the container with id 80251af015d263cb4babcda9bdba2af729e5e005cafd0813bb616aa2ff925eca Jul 21 18:01:38.767914 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:38.767878 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae64415c_2f98_4d23_854d_adc8fef068c9.slice/crio-504c201163a51fd33ce372552584fe724a58aba91b7fa96ee9b0b80d04d5cbe2 WatchSource:0}: Error finding container 504c201163a51fd33ce372552584fe724a58aba91b7fa96ee9b0b80d04d5cbe2: Status 404 returned error can't find the container with id 504c201163a51fd33ce372552584fe724a58aba91b7fa96ee9b0b80d04d5cbe2 Jul 21 18:01:38.851298 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.851055 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" event={"ID":"ee88fd76-5152-4601-a70a-b041335879d1","Type":"ContainerStarted","Data":"6fe1929894b23d6e5eb46f439b50276b5d913b876922b1ebfe526e1e160c28c7"} Jul 21 18:01:38.851298 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.851129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" event={"ID":"ee88fd76-5152-4601-a70a-b041335879d1","Type":"ContainerStarted","Data":"1937069b5d1858558cf8864eda4b019097bb073f79d53f22be08245b363f7967"} Jul 21 18:01:38.851298 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.851159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" event={"ID":"ee88fd76-5152-4601-a70a-b041335879d1","Type":"ContainerStarted","Data":"02c949300c74d9faf4f975d3cc26208fc25e397977559ad92e89b6d4c715fb3a"} Jul 21 18:01:38.853159 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.853133 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555dc88856-fbbxs" event={"ID":"3e51ba0f-1598-4e5c-b9ac-3e028b475e61","Type":"ContainerStarted","Data":"74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4"} Jul 21 18:01:38.853282 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.853181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555dc88856-fbbxs" event={"ID":"3e51ba0f-1598-4e5c-b9ac-3e028b475e61","Type":"ContainerStarted","Data":"80251af015d263cb4babcda9bdba2af729e5e005cafd0813bb616aa2ff925eca"} Jul 21 18:01:38.856328 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.856260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" event={"ID":"ae64415c-2f98-4d23-854d-adc8fef068c9","Type":"ContainerStarted","Data":"504c201163a51fd33ce372552584fe724a58aba91b7fa96ee9b0b80d04d5cbe2"} Jul 21 18:01:38.858231 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.858204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"b2d26fb4c6acbff3ce852fb25165c5c614e992f31a8a836591ecdccacf76d893"} Jul 21 18:01:38.859917 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.859895 2560 generic.go:356] "Generic (PLEG): container finished" podID="45beceea-a768-4f04-a69a-e54e0431fea6" containerID="1569020aee37a9ff78c90ff43afff1aa93d3a0434b8930bba3671b43165e07f4" exitCode=0 Jul 21 18:01:38.860062 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.860037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sz72g" event={"ID":"45beceea-a768-4f04-a69a-e54e0431fea6","Type":"ContainerDied","Data":"1569020aee37a9ff78c90ff43afff1aa93d3a0434b8930bba3671b43165e07f4"} Jul 21 18:01:38.861698 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.861675 2560 generic.go:356] "Generic (PLEG): container finished" podID="f1448cc9-0af4-47aa-86f3-9e13c8a45f25" containerID="4b522c6f939ced6d1b3df3bacbc1a3c224b0ad705f7051ba1a8da06e7f25ed5e" exitCode=0 Jul 21 18:01:38.861784 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.861738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerDied","Data":"4b522c6f939ced6d1b3df3bacbc1a3c224b0ad705f7051ba1a8da06e7f25ed5e"} Jul 21 18:01:38.864080 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.864057 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" event={"ID":"d2069994-290e-4e72-a12f-8e509e333ca7","Type":"ContainerStarted","Data":"c280ee76f5333fb608af9f22550b00e2a237ab639743e17dc33ff5920712a8df"} Jul 21 18:01:38.866220 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.866198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" event={"ID":"80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181","Type":"ContainerStarted","Data":"a9433686fb30b99ec215e2dc75574c8ea61938d1d0b49bbc7c49d2630a918afc"} Jul 21 18:01:38.901973 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.901879 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-rzn86" podStartSLOduration=3.44029086 podStartE2EDuration="5.901855589s" totalImagesPullingTime="2.461564729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:38.87284343 +0000 UTC m=+73.945439129" watchObservedRunningTime="2026-07-21 18:01:38.901855589 +0000 UTC m=+73.974451290" Jul 21 18:01:38.969408 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.968631 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zznq7" podStartSLOduration=3.623414448 podStartE2EDuration="5.968609937s" totalImagesPullingTime="2.345195489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:38.95781492 +0000 UTC m=+74.030410611" watchObservedRunningTime="2026-07-21 18:01:38.968609937 +0000 UTC m=+74.041205638" Jul 21 18:01:38.989263 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:38.989211 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-555dc88856-fbbxs" podStartSLOduration=0.989194617 podStartE2EDuration="989.194617ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:38.987069879 +0000 UTC m=+74.059665576" watchObservedRunningTime="2026-07-21 18:01:38.989194617 +0000 UTC m=+74.061790294" Jul 21 18:01:39.768703 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.768667 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 18:01:39.775164 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.775139 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.780106 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.780084 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 21 18:01:39.780246 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.780172 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 21 18:01:39.780246 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.780193 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 21 18:01:39.780373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.780316 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 21 18:01:39.781584 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.781101 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 21 18:01:39.781584 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.781315 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sp4p2\"" Jul 21 18:01:39.781584 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.781519 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 21 18:01:39.781789 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.781616 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 21 18:01:39.781789 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.781677 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dejc5js70822u\"" Jul 21 18:01:39.784782 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.784477 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 21 18:01:39.784782 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.784647 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 21 18:01:39.785639 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.785607 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 21 18:01:39.785844 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.785828 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 21 18:01:39.786096 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.786073 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 21 18:01:39.789238 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.789220 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 21 18:01:39.792988 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.792967 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 21 18:01:39.809951 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.809922 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 18:01:39.838003 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.837972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838201 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838011 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838201 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838201 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838201 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838253 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838380 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838403 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vhp2\" (UniqueName: \"kubernetes.io/projected/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-kube-api-access-6vhp2\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838535 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838574 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838625 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-config\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.838654 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.838648 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.939940 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.939905 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-config\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.939981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940031 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940078 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.940430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vhp2\" (UniqueName: \"kubernetes.io/projected/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-kube-api-access-6vhp2\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.940678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.941927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.941968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.942009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.942177 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.942945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.942888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.943361 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.943328 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.943711 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.943650 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.945029 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.945395 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.945707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.945797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.945825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.945852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.946126 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946244 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.946181 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-config\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.946661 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.946586 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.947017 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.946995 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.947212 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.947192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.947269 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.947254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.948285 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.948268 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.948940 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.948910 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.949156 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.949114 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:39.969225 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:39.969186 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vhp2\" (UniqueName: \"kubernetes.io/projected/5f818b7c-d9b2-446d-a6d4-d38c6cf167b5-kube-api-access-6vhp2\") pod \"prometheus-k8s-0\" (UID: \"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:40.090730 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.090633 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:40.639830 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.639798 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 18:01:40.642791 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:01:40.642752 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f818b7c_d9b2_446d_a6d4_d38c6cf167b5.slice/crio-a1a00e40359e28704cf6e71195fb8e2e13873609a5e331f6fc504af311941374 WatchSource:0}: Error finding container a1a00e40359e28704cf6e71195fb8e2e13873609a5e331f6fc504af311941374: Status 404 returned error can't find the container with id a1a00e40359e28704cf6e71195fb8e2e13873609a5e331f6fc504af311941374 Jul 21 18:01:40.879213 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.879177 2560 generic.go:356] "Generic (PLEG): container finished" podID="5f818b7c-d9b2-446d-a6d4-d38c6cf167b5" containerID="ba8e154dd2ceee6a7ebf4149e1394e061600e389943f730c5db5a5470597bacc" exitCode=0 Jul 21 18:01:40.879391 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.879303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerDied","Data":"ba8e154dd2ceee6a7ebf4149e1394e061600e389943f730c5db5a5470597bacc"} Jul 21 18:01:40.879391 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.879378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"a1a00e40359e28704cf6e71195fb8e2e13873609a5e331f6fc504af311941374"} Jul 21 18:01:40.884685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.884656 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sz72g" event={"ID":"45beceea-a768-4f04-a69a-e54e0431fea6","Type":"ContainerStarted","Data":"91419d2b1241de159c261974607f1568dd82bd07a4445bd71953bd6b6b9ae727"} Jul 21 18:01:40.884778 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.884714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sz72g" event={"ID":"45beceea-a768-4f04-a69a-e54e0431fea6","Type":"ContainerStarted","Data":"8d9f3531cbd70d037103f1655975e3e96808a081330276fff418481d85e53dbc"} Jul 21 18:01:40.888700 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.888666 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kzvc2" event={"ID":"ad0416fc-678d-4d6d-8715-24d76b208eb5","Type":"ContainerStarted","Data":"f4b4134c1c85203088a280f3008f33d40f9b5536d8e1e32d0334cb62b92c52aa"} Jul 21 18:01:40.932404 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.932346 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kzvc2" podStartSLOduration=66.100892572 podStartE2EDuration="1m14.932309713s" totalImagesPullingTime="8.831417141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:40.929921471 +0000 UTC m=+76.002517169" watchObservedRunningTime="2026-07-21 18:01:40.932309713 +0000 UTC m=+76.004905415" Jul 21 18:01:40.953669 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:40.953605 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sz72g" podStartSLOduration=5.066826748 podStartE2EDuration="7.953582779s" totalImagesPullingTime="2.730475888s" totalInitContainerRuntime="156.280143ms" isStatefulPod=true podCreationTimestamp="2026-07-21 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:40.951106396 +0000 UTC m=+76.023702095" watchObservedRunningTime="2026-07-21 18:01:40.953582779 +0000 UTC m=+76.026178478" Jul 21 18:01:43.906738 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.906698 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" event={"ID":"ae64415c-2f98-4d23-854d-adc8fef068c9","Type":"ContainerStarted","Data":"394f61e98d06621378a1edda5c5034e82e30d829d9b7cca46c1e431932eaceb4"} Jul 21 18:01:43.907201 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.906900 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:43.908934 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.908903 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" event={"ID":"d2069994-290e-4e72-a12f-8e509e333ca7","Type":"ContainerStarted","Data":"7550faf80fe5663e928c9c194603bee327fb1c9a06d6598525372450092e9cbf"} Jul 21 18:01:43.911281 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.911250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"a7cccfd6e9329dcc3693a6106e7bdf37835ad7c0b5eaa99a57de48abb110bbb6"} Jul 21 18:01:43.911389 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.911326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"cb7507242a25264f4fe88282bfa2d482e19af3e25ee15a611efa02c581f940be"} Jul 21 18:01:43.913481 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.913459 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" Jul 21 18:01:43.913713 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.913692 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"54cfcaa84c5166a3842d90b27af78432a1094c1f5eb9c16d389d84eaaadd3a2e"} Jul 21 18:01:43.913776 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.913746 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"7dca69a82772a09a729da686aee3ecfc0ee7c2ba5e7655db3c7e9f2cded264ea"} Jul 21 18:01:43.926854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.926814 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qnmtc" podStartSLOduration=1.451436975 podStartE2EDuration="5.92680136s" totalImagesPullingTime="4.475364385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:43.925797986 +0000 UTC m=+78.998393686" watchObservedRunningTime="2026-07-21 18:01:43.92680136 +0000 UTC m=+78.999397058" Jul 21 18:01:43.966821 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:43.966760 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" podStartSLOduration=2.354987846 podStartE2EDuration="6.966740624s" totalImagesPullingTime="4.611752778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:43.964532135 +0000 UTC m=+79.037127831" watchObservedRunningTime="2026-07-21 18:01:43.966740624 +0000 UTC m=+79.039336330" Jul 21 18:01:44.921970 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.921939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"c4106db30902b7208b15b3e3e66759a65571ec128c8691da5e838a1282b4aeca"} Jul 21 18:01:44.922430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.922032 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"d9ea847b0214261d46e674f39385924256f3f0384354e3023e340ceab7c59bd3"} Jul 21 18:01:44.922430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.922059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"dd8b91dfefb6d8156b9ffc4f9e2ce5eafd022af4919095e704e48a8ac4ae3e58"} Jul 21 18:01:44.924186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.924160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"23d6a28fe1de9a9f357745a2a555d7b1c9bffd446c1aa3c5643aec854999c83f"} Jul 21 18:01:44.927634 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.927595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"1dbe9c700bb8b434b2702f3cf8daa5a994ebbace5ccc49ac6f84be728e2baeb5"} Jul 21 18:01:44.927747 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.927689 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"09056f5bc4ae56154b66cab0d407df68e742145e41febef52f7d8cce20a938c7"} Jul 21 18:01:44.927747 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:44.927736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"98d880932ac2e45872c0ede9fb82c738c9b12a582767bbf5c177498757b80217"} Jul 21 18:01:45.935774 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.935738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"ca8999c5c9fbf3455d26ffe94ade9e76b25bcedbb89e489bba1c3b1e4f44e8b2"} Jul 21 18:01:45.936226 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.935824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"c0e98d502a87f266e4ac3590248d5b5d64183aef41fb4ad6a54340a98577228f"} Jul 21 18:01:45.936226 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.935867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f818b7c-d9b2-446d-a6d4-d38c6cf167b5","Type":"ContainerStarted","Data":"272f23b8df75832d745686b24ba5fa699d5efa419d8c11ab10a3262366af9c2e"} Jul 21 18:01:45.938995 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.938960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"4d5e46108a3bca89ca84527a96cda82a20d33916527d54c0145833bcab0e54ae"} Jul 21 18:01:45.939118 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.939028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"5077f757bd3e5266d509143e220c603e68ae11213ea066c622eb7ba0e8cdfc1a"} Jul 21 18:01:45.939118 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.939078 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:45.939235 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.939120 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" event={"ID":"9536aa4a-a08d-47dc-a51e-b3128755c993","Type":"ContainerStarted","Data":"930a0e8fbd19a8e71ad61c923f14a9f887fbaf047a5e16acd0e13cce505ea88c"} Jul 21 18:01:45.942393 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.942368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1448cc9-0af4-47aa-86f3-9e13c8a45f25","Type":"ContainerStarted","Data":"5348886515c3b3844d594277f6f8652f7c4c92b4a4a268c061aaeb006db6fda6"} Jul 21 18:01:45.970085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.970033 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.188745704 podStartE2EDuration="6.970017889s" totalImagesPullingTime="3.746501811s" totalInitContainerRuntime="34.770374ms" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:45.967217323 +0000 UTC m=+81.039813021" watchObservedRunningTime="2026-07-21 18:01:45.970017889 +0000 UTC m=+81.042613587" Jul 21 18:01:45.992227 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:45.992181 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" podStartSLOduration=4.257014922 podStartE2EDuration="9.992167575s" totalImagesPullingTime="5.735152653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:45.990330874 +0000 UTC m=+81.062926582" watchObservedRunningTime="2026-07-21 18:01:45.992167575 +0000 UTC m=+81.064763311" Jul 21 18:01:46.020613 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:46.020572 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.376510948 podStartE2EDuration="12.020556742s" totalImagesPullingTime="7.605330944s" totalInitContainerRuntime="38.71485ms" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:46.018633439 +0000 UTC m=+81.091229179" watchObservedRunningTime="2026-07-21 18:01:46.020556742 +0000 UTC m=+81.093152440" Jul 21 18:01:48.481098 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:48.481067 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:48.481724 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:48.481165 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:48.486065 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:48.486036 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:48.958748 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:48.958720 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:01:49.017876 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:49.017836 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7555b687-j62vv"] Jul 21 18:01:50.091078 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:50.091043 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:01:51.953729 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:51.953701 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78b4658575-flbgw" Jul 21 18:01:58.114641 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:58.114601 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:01:58.115024 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:01:58.114674 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:02:06.842609 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:06.842574 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2cnsl" Jul 21 18:02:14.043928 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.043866 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b7555b687-j62vv" podUID="28838bed-da16-479e-9f52-a880c48de27b" containerName="console" containerID="cri-o://a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e" gracePeriod=15 Jul 21 18:02:14.287648 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.287619 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:02:14.360578 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360475 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-oauth-serving-cert\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.360578 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360515 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-trusted-ca-bundle\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.360578 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360553 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-console-config\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.360854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360703 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-oauth-config\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.360854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360760 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-serving-cert\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.360854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360802 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-service-ca\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.360854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360847 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q4bx\" (UniqueName: \"kubernetes.io/projected/28838bed-da16-479e-9f52-a880c48de27b-kube-api-access-4q4bx\") pod \"28838bed-da16-479e-9f52-a880c48de27b\" (UID: \"28838bed-da16-479e-9f52-a880c48de27b\") " Jul 21 18:02:14.361049 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.360965 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:02:14.361049 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.361004 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-console-config" (OuterVolumeSpecName: "console-config") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:02:14.361174 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.361148 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:02:14.361402 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.361351 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-console-config\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:14.361519 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.361364 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-service-ca" (OuterVolumeSpecName: "service-ca") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:02:14.361580 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.361525 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-trusted-ca-bundle\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:14.363339 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.363306 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28838bed-da16-479e-9f52-a880c48de27b-kube-api-access-4q4bx" (OuterVolumeSpecName: "kube-api-access-4q4bx") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "kube-api-access-4q4bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:02:14.363470 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.363337 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:02:14.363623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.363596 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "28838bed-da16-479e-9f52-a880c48de27b" (UID: "28838bed-da16-479e-9f52-a880c48de27b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:02:14.462396 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.462349 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q4bx\" (UniqueName: \"kubernetes.io/projected/28838bed-da16-479e-9f52-a880c48de27b-kube-api-access-4q4bx\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:14.462396 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.462378 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-oauth-serving-cert\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:14.462396 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.462388 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-oauth-config\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:14.462396 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.462396 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28838bed-da16-479e-9f52-a880c48de27b-console-serving-cert\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:14.462396 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:14.462404 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28838bed-da16-479e-9f52-a880c48de27b-service-ca\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:02:15.063685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.063651 2560 generic.go:356] "Generic (PLEG): container finished" podID="28838bed-da16-479e-9f52-a880c48de27b" containerID="a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e" exitCode=0 Jul 21 18:02:15.064083 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.063704 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7555b687-j62vv" Jul 21 18:02:15.064083 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.063721 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7555b687-j62vv" event={"ID":"28838bed-da16-479e-9f52-a880c48de27b","Type":"ContainerDied","Data":"a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e"} Jul 21 18:02:15.064083 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.063775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7555b687-j62vv" event={"ID":"28838bed-da16-479e-9f52-a880c48de27b","Type":"ContainerDied","Data":"6265fff41c887939ddc2d8cc40f60d5870377c4be80467b5e4fc6165ff959897"} Jul 21 18:02:15.064083 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.063801 2560 scope.go:117] "RemoveContainer" containerID="a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e" Jul 21 18:02:15.072579 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.072561 2560 scope.go:117] "RemoveContainer" containerID="a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e" Jul 21 18:02:15.072850 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:02:15.072829 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e\": container with ID starting with a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e not found: ID does not exist" containerID="a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e" Jul 21 18:02:15.072912 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.072859 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e"} err="failed to get container status \"a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e\": rpc error: code = NotFound desc = could not find container \"a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e\": container with ID starting with a086635493961c28c0ef1f2e904e5f80610ececd411d179e1a935d4a441ed09e not found: ID does not exist" Jul 21 18:02:15.086143 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.086115 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7555b687-j62vv"] Jul 21 18:02:15.092023 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.092002 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7555b687-j62vv"] Jul 21 18:02:15.532589 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:15.532550 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28838bed-da16-479e-9f52-a880c48de27b" path="/var/lib/kubelet/pods/28838bed-da16-479e-9f52-a880c48de27b/volumes" Jul 21 18:02:18.119670 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:18.119633 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:02:18.123824 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:18.123798 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c5b88859d-g2wgs" Jul 21 18:02:40.091724 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:40.091688 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:02:40.113172 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:40.113142 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:02:40.185347 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:02:40.185319 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 18:03:14.217806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:14.217768 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-555dc88856-fbbxs"] Jul 21 18:03:39.319630 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.319585 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-555dc88856-fbbxs" podUID="3e51ba0f-1598-4e5c-b9ac-3e028b475e61" containerName="console" containerID="cri-o://74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4" gracePeriod=15 Jul 21 18:03:39.568122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.568097 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:03:39.702206 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702170 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-config\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702405 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702232 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-trusted-ca-bundle\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702405 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702267 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98mnh\" (UniqueName: \"kubernetes.io/projected/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-kube-api-access-98mnh\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702405 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702295 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-service-ca\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702405 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702335 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-oauth-config\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702409 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-oauth-serving-cert\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702432 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-serving-cert\") pod \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\" (UID: \"3e51ba0f-1598-4e5c-b9ac-3e028b475e61\") " Jul 21 18:03:39.702899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702656 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-config" (OuterVolumeSpecName: "console-config") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:39.702899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702726 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:39.702899 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702738 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:39.703104 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.702986 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:39.704854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.704827 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-kube-api-access-98mnh" (OuterVolumeSpecName: "kube-api-access-98mnh") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "kube-api-access-98mnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:03:39.704854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.704827 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:03:39.704974 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.704893 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e51ba0f-1598-4e5c-b9ac-3e028b475e61" (UID: "3e51ba0f-1598-4e5c-b9ac-3e028b475e61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:03:39.804075 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804033 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-config\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:39.804075 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804067 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-trusted-ca-bundle\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:39.804075 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804077 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98mnh\" (UniqueName: \"kubernetes.io/projected/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-kube-api-access-98mnh\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:39.804075 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804087 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-service-ca\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:39.804348 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804095 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-oauth-config\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:39.804348 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804104 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-oauth-serving-cert\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:39.804348 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:39.804112 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e51ba0f-1598-4e5c-b9ac-3e028b475e61-console-serving-cert\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:03:40.404258 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.404229 2560 generic.go:356] "Generic (PLEG): container finished" podID="3e51ba0f-1598-4e5c-b9ac-3e028b475e61" containerID="74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4" exitCode=0 Jul 21 18:03:40.404685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.404275 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-555dc88856-fbbxs" Jul 21 18:03:40.404685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.404315 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555dc88856-fbbxs" event={"ID":"3e51ba0f-1598-4e5c-b9ac-3e028b475e61","Type":"ContainerDied","Data":"74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4"} Jul 21 18:03:40.404685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.404365 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-555dc88856-fbbxs" event={"ID":"3e51ba0f-1598-4e5c-b9ac-3e028b475e61","Type":"ContainerDied","Data":"80251af015d263cb4babcda9bdba2af729e5e005cafd0813bb616aa2ff925eca"} Jul 21 18:03:40.404685 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.404393 2560 scope.go:117] "RemoveContainer" containerID="74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4" Jul 21 18:03:40.419228 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.419209 2560 scope.go:117] "RemoveContainer" containerID="74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4" Jul 21 18:03:40.419527 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:03:40.419506 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4\": container with ID starting with 74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4 not found: ID does not exist" containerID="74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4" Jul 21 18:03:40.419602 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.419535 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4"} err="failed to get container status \"74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4\": rpc error: code = NotFound desc = could not find container \"74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4\": container with ID starting with 74b87dc1538576050c26bbe8566fd2c41cf2412a2bab647dc19588e778dcadf4 not found: ID does not exist" Jul 21 18:03:40.431750 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.431726 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-555dc88856-fbbxs"] Jul 21 18:03:40.437900 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:40.437872 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-555dc88856-fbbxs"] Jul 21 18:03:41.531922 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:03:41.531886 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e51ba0f-1598-4e5c-b9ac-3e028b475e61" path="/var/lib/kubelet/pods/3e51ba0f-1598-4e5c-b9ac-3e028b475e61/volumes" Jul 21 18:04:49.283732 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.283650 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8"] Jul 21 18:04:49.284145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.283954 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e51ba0f-1598-4e5c-b9ac-3e028b475e61" containerName="console" Jul 21 18:04:49.284145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.283965 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e51ba0f-1598-4e5c-b9ac-3e028b475e61" containerName="console" Jul 21 18:04:49.284145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.283979 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28838bed-da16-479e-9f52-a880c48de27b" containerName="console" Jul 21 18:04:49.284145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.283985 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28838bed-da16-479e-9f52-a880c48de27b" containerName="console" Jul 21 18:04:49.284145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.284039 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="28838bed-da16-479e-9f52-a880c48de27b" containerName="console" Jul 21 18:04:49.284145 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.284047 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e51ba0f-1598-4e5c-b9ac-3e028b475e61" containerName="console" Jul 21 18:04:49.286715 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.286698 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.290253 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.290233 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-r7pmf\"" Jul 21 18:04:49.291432 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.291417 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 18:04:49.291510 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.291482 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 18:04:49.297716 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.297697 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8"] Jul 21 18:04:49.387256 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.387222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9ctv\" (UniqueName: \"kubernetes.io/projected/807b0ddf-a8a8-44b5-9b2a-71493fef083f-kube-api-access-b9ctv\") pod \"cert-manager-cainjector-559cdd8c87-5s8g8\" (UID: \"807b0ddf-a8a8-44b5-9b2a-71493fef083f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.387256 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.387265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/807b0ddf-a8a8-44b5-9b2a-71493fef083f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5s8g8\" (UID: \"807b0ddf-a8a8-44b5-9b2a-71493fef083f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.493586 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.493408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9ctv\" (UniqueName: \"kubernetes.io/projected/807b0ddf-a8a8-44b5-9b2a-71493fef083f-kube-api-access-b9ctv\") pod \"cert-manager-cainjector-559cdd8c87-5s8g8\" (UID: \"807b0ddf-a8a8-44b5-9b2a-71493fef083f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.493768 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.493635 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/807b0ddf-a8a8-44b5-9b2a-71493fef083f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5s8g8\" (UID: \"807b0ddf-a8a8-44b5-9b2a-71493fef083f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.503087 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.503055 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/807b0ddf-a8a8-44b5-9b2a-71493fef083f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5s8g8\" (UID: \"807b0ddf-a8a8-44b5-9b2a-71493fef083f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.503233 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.503214 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9ctv\" (UniqueName: \"kubernetes.io/projected/807b0ddf-a8a8-44b5-9b2a-71493fef083f-kube-api-access-b9ctv\") pod \"cert-manager-cainjector-559cdd8c87-5s8g8\" (UID: \"807b0ddf-a8a8-44b5-9b2a-71493fef083f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.607744 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.607663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" Jul 21 18:04:49.737322 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:49.737299 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8"] Jul 21 18:04:49.739510 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:04:49.739483 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod807b0ddf_a8a8_44b5_9b2a_71493fef083f.slice/crio-2946dfbd394357f5f6339255546d1554f34f9be8da69665266c4453eaa5a9aa1 WatchSource:0}: Error finding container 2946dfbd394357f5f6339255546d1554f34f9be8da69665266c4453eaa5a9aa1: Status 404 returned error can't find the container with id 2946dfbd394357f5f6339255546d1554f34f9be8da69665266c4453eaa5a9aa1 Jul 21 18:04:50.689368 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:50.689326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" event={"ID":"807b0ddf-a8a8-44b5-9b2a-71493fef083f","Type":"ContainerStarted","Data":"2946dfbd394357f5f6339255546d1554f34f9be8da69665266c4453eaa5a9aa1"} Jul 21 18:04:53.704063 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:53.704017 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" event={"ID":"807b0ddf-a8a8-44b5-9b2a-71493fef083f","Type":"ContainerStarted","Data":"29e79277410aa5e6476e747c7380618045377daa6b0163b1b7b40b61286f09e1"} Jul 21 18:04:53.721056 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:04:53.721001 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s8g8" podStartSLOduration=1.432134701 podStartE2EDuration="4.720979984s" totalImagesPullingTime="3.288845283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:04:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:04:53.720176648 +0000 UTC m=+268.792772328" watchObservedRunningTime="2026-07-21 18:04:53.720979984 +0000 UTC m=+268.793575681" Jul 21 18:05:23.101527 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.101495 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp"] Jul 21 18:05:23.103663 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.103647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.106139 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.106108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6rfkl\"" Jul 21 18:05:23.106301 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.106283 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:05:23.107073 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.107055 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 18:05:23.115183 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.115162 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp"] Jul 21 18:05:23.193591 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.193556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1bd2d95-6a07-45e8-a6cd-b9938516b24c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fz6tp\" (UID: \"e1bd2d95-6a07-45e8-a6cd-b9938516b24c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.193740 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.193614 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgfcp\" (UniqueName: \"kubernetes.io/projected/e1bd2d95-6a07-45e8-a6cd-b9938516b24c-kube-api-access-pgfcp\") pod \"openshift-lws-operator-fd8ccff4c-fz6tp\" (UID: \"e1bd2d95-6a07-45e8-a6cd-b9938516b24c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.294045 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.294001 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1bd2d95-6a07-45e8-a6cd-b9938516b24c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fz6tp\" (UID: \"e1bd2d95-6a07-45e8-a6cd-b9938516b24c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.294215 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.294079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgfcp\" (UniqueName: \"kubernetes.io/projected/e1bd2d95-6a07-45e8-a6cd-b9938516b24c-kube-api-access-pgfcp\") pod \"openshift-lws-operator-fd8ccff4c-fz6tp\" (UID: \"e1bd2d95-6a07-45e8-a6cd-b9938516b24c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.294490 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.294437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1bd2d95-6a07-45e8-a6cd-b9938516b24c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fz6tp\" (UID: \"e1bd2d95-6a07-45e8-a6cd-b9938516b24c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.303398 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.303362 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgfcp\" (UniqueName: \"kubernetes.io/projected/e1bd2d95-6a07-45e8-a6cd-b9938516b24c-kube-api-access-pgfcp\") pod \"openshift-lws-operator-fd8ccff4c-fz6tp\" (UID: \"e1bd2d95-6a07-45e8-a6cd-b9938516b24c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.412468 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.412416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" Jul 21 18:05:23.545874 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.545852 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp"] Jul 21 18:05:23.548373 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:05:23.548347 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1bd2d95_6a07_45e8_a6cd_b9938516b24c.slice/crio-3845ad73944611309f3c8caf154aab4d7fe58c6e41e7a9927640df0d605acc46 WatchSource:0}: Error finding container 3845ad73944611309f3c8caf154aab4d7fe58c6e41e7a9927640df0d605acc46: Status 404 returned error can't find the container with id 3845ad73944611309f3c8caf154aab4d7fe58c6e41e7a9927640df0d605acc46 Jul 21 18:05:23.820040 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:23.819957 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" event={"ID":"e1bd2d95-6a07-45e8-a6cd-b9938516b24c","Type":"ContainerStarted","Data":"3845ad73944611309f3c8caf154aab4d7fe58c6e41e7a9927640df0d605acc46"} Jul 21 18:05:25.647825 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:25.647794 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:05:25.649267 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:25.648034 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:05:25.652508 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:25.652491 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 18:05:25.830282 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:25.830240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" event={"ID":"e1bd2d95-6a07-45e8-a6cd-b9938516b24c","Type":"ContainerStarted","Data":"ca589f767f87cf267ea81a4cdf2623dd3a88a5bc9a999baede4266e3a165e357"} Jul 21 18:05:25.851500 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:25.851421 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fz6tp" podStartSLOduration=0.736069573 podStartE2EDuration="2.851401786s" totalImagesPullingTime="2.115332213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:05:25.849166904 +0000 UTC m=+300.921762856" watchObservedRunningTime="2026-07-21 18:05:25.851401786 +0000 UTC m=+300.923997485" Jul 21 18:05:33.044205 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.044153 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms"] Jul 21 18:05:33.046767 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.046746 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.049258 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.049232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 18:05:33.049390 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.049233 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 18:05:33.049390 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.049241 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 18:05:33.049627 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.049614 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4m2hr\"" Jul 21 18:05:33.065742 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.065718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms"] Jul 21 18:05:33.081274 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.081241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjxm\" (UniqueName: \"kubernetes.io/projected/e6631fc4-3caf-4b06-85c2-3df49f258357-kube-api-access-shjxm\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.081425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.081317 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e6631fc4-3caf-4b06-85c2-3df49f258357-manager-config\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.081425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.081359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6631fc4-3caf-4b06-85c2-3df49f258357-metrics-cert\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.081425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.081375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6631fc4-3caf-4b06-85c2-3df49f258357-cert\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.182095 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.182060 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjxm\" (UniqueName: \"kubernetes.io/projected/e6631fc4-3caf-4b06-85c2-3df49f258357-kube-api-access-shjxm\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.182270 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.182116 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e6631fc4-3caf-4b06-85c2-3df49f258357-manager-config\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.182270 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.182148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6631fc4-3caf-4b06-85c2-3df49f258357-metrics-cert\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.182270 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.182166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6631fc4-3caf-4b06-85c2-3df49f258357-cert\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.182951 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.182924 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e6631fc4-3caf-4b06-85c2-3df49f258357-manager-config\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.184732 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.184702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6631fc4-3caf-4b06-85c2-3df49f258357-cert\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.184821 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.184748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6631fc4-3caf-4b06-85c2-3df49f258357-metrics-cert\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.192484 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.192440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjxm\" (UniqueName: \"kubernetes.io/projected/e6631fc4-3caf-4b06-85c2-3df49f258357-kube-api-access-shjxm\") pod \"lws-controller-manager-754c46d99f-bvfms\" (UID: \"e6631fc4-3caf-4b06-85c2-3df49f258357\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.356191 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.356078 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:33.495484 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.495366 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms"] Jul 21 18:05:33.498175 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:05:33.498126 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6631fc4_3caf_4b06_85c2_3df49f258357.slice/crio-a4aaefdbb85b564a838b3cd03f1d0568ac536ef8abda223e619e564932a5d156 WatchSource:0}: Error finding container a4aaefdbb85b564a838b3cd03f1d0568ac536ef8abda223e619e564932a5d156: Status 404 returned error can't find the container with id a4aaefdbb85b564a838b3cd03f1d0568ac536ef8abda223e619e564932a5d156 Jul 21 18:05:33.500059 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.500042 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:05:33.863918 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:33.863875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" event={"ID":"e6631fc4-3caf-4b06-85c2-3df49f258357","Type":"ContainerStarted","Data":"a4aaefdbb85b564a838b3cd03f1d0568ac536ef8abda223e619e564932a5d156"} Jul 21 18:05:35.873413 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:35.873376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" event={"ID":"e6631fc4-3caf-4b06-85c2-3df49f258357","Type":"ContainerStarted","Data":"62ddbdb895402fb2829db039e10b9f8da4123ecf2afd05db08a9ebb204630aa8"} Jul 21 18:05:35.873840 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:35.873519 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:42.247334 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.247270 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" podStartSLOduration=7.86707173 podStartE2EDuration="9.247253379s" totalImagesPullingTime="1.380181649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:05:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:05:35.914127175 +0000 UTC m=+310.986722871" watchObservedRunningTime="2026-07-21 18:05:42.247253379 +0000 UTC m=+317.319849076" Jul 21 18:05:42.248521 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.248497 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h"] Jul 21 18:05:42.251026 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.251005 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.254945 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.254910 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 18:05:42.255585 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.255567 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 18:05:42.257323 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.257287 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 18:05:42.258929 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.258175 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 18:05:42.259718 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.259700 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7tzw6\"" Jul 21 18:05:42.298601 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.298567 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h"] Jul 21 18:05:42.370521 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.370481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.370695 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.370542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.370695 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.370569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7pzk\" (UniqueName: \"kubernetes.io/projected/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-kube-api-access-w7pzk\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.370695 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.370590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.370808 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.370714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.471886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.471843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.472110 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.471914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.472110 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.471961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.472110 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.471988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.472110 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.472009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7pzk\" (UniqueName: \"kubernetes.io/projected/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-kube-api-access-w7pzk\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.472390 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.472365 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.474492 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.474440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.474616 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.474601 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.474675 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.474609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.485400 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.485364 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7pzk\" (UniqueName: \"kubernetes.io/projected/06d343d6-f9b6-472d-ab2a-b1c661ff50e1-kube-api-access-w7pzk\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-pb58h\" (UID: \"06d343d6-f9b6-472d-ab2a-b1c661ff50e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.563929 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.563835 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:42.727604 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.727563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h"] Jul 21 18:05:42.730742 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:05:42.730714 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d343d6_f9b6_472d_ab2a_b1c661ff50e1.slice/crio-f9b39fa83a3a0e13be4fdd03987a38580799b5006e7b6543c8e9351de3311ba1 WatchSource:0}: Error finding container f9b39fa83a3a0e13be4fdd03987a38580799b5006e7b6543c8e9351de3311ba1: Status 404 returned error can't find the container with id f9b39fa83a3a0e13be4fdd03987a38580799b5006e7b6543c8e9351de3311ba1 Jul 21 18:05:42.905006 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:42.904963 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" event={"ID":"06d343d6-f9b6-472d-ab2a-b1c661ff50e1","Type":"ContainerStarted","Data":"f9b39fa83a3a0e13be4fdd03987a38580799b5006e7b6543c8e9351de3311ba1"} Jul 21 18:05:45.923519 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:45.923407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" event={"ID":"06d343d6-f9b6-472d-ab2a-b1c661ff50e1","Type":"ContainerStarted","Data":"d0f7fc0b30a274960de2bafb8be7af7bf5fb405730dedb87e7d0e9194d535604"} Jul 21 18:05:46.879719 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:46.879688 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-bvfms" Jul 21 18:05:46.930465 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:46.930409 2560 generic.go:356] "Generic (PLEG): container finished" podID="06d343d6-f9b6-472d-ab2a-b1c661ff50e1" containerID="d0f7fc0b30a274960de2bafb8be7af7bf5fb405730dedb87e7d0e9194d535604" exitCode=0 Jul 21 18:05:46.930897 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:46.930507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" event={"ID":"06d343d6-f9b6-472d-ab2a-b1c661ff50e1","Type":"ContainerDied","Data":"d0f7fc0b30a274960de2bafb8be7af7bf5fb405730dedb87e7d0e9194d535604"} Jul 21 18:05:47.941267 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:47.941226 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" event={"ID":"06d343d6-f9b6-472d-ab2a-b1c661ff50e1","Type":"ContainerStarted","Data":"7e20f78fde2efab4141331942c91439f2ab32bb985512dacf70248095b1a9dba"} Jul 21 18:05:47.941720 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:47.941342 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:05:47.973113 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:47.973043 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" podStartSLOduration=1.661647206 podStartE2EDuration="5.973019377s" totalImagesPullingTime="3.257273075s" totalInitContainerRuntime="1.054099096s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:05:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:05:47.971333285 +0000 UTC m=+323.043928985" watchObservedRunningTime="2026-07-21 18:05:47.973019377 +0000 UTC m=+323.045615076" Jul 21 18:05:58.948585 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:05:58.948551 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-pb58h" Jul 21 18:06:01.726060 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.726019 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg"] Jul 21 18:06:01.860926 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.860886 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg"] Jul 21 18:06:01.861091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.861015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:01.865896 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.865866 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 18:06:01.866027 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.865924 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 18:06:01.866027 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.865877 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8xbnh\"" Jul 21 18:06:01.866739 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.866720 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 18:06:01.873232 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.873211 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 18:06:01.952032 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.951994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq2wt\" (UniqueName: \"kubernetes.io/projected/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-kube-api-access-gq2wt\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:01.952227 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.952055 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-tls-certs\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:01.952227 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:01.952122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-tmp\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.053634 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.053538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-tls-certs\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.053634 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.053576 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-tmp\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.053827 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.053637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq2wt\" (UniqueName: \"kubernetes.io/projected/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-kube-api-access-gq2wt\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.056163 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.056109 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-tmp\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.056335 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.056244 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-tls-certs\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.066681 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.066645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq2wt\" (UniqueName: \"kubernetes.io/projected/9fd43684-6823-40bd-9c6f-ebabf4bfb8f1-kube-api-access-gq2wt\") pod \"kube-auth-proxy-f49f77f5f-nwjwg\" (UID: \"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.171372 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.171322 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" Jul 21 18:06:02.343770 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:02.343684 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg"] Jul 21 18:06:02.346152 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:06:02.346114 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd43684_6823_40bd_9c6f_ebabf4bfb8f1.slice/crio-c9e90a319f8294e4fb1e0be62bf471307051eaa17000d31d1dfbb970e997dff5 WatchSource:0}: Error finding container c9e90a319f8294e4fb1e0be62bf471307051eaa17000d31d1dfbb970e997dff5: Status 404 returned error can't find the container with id c9e90a319f8294e4fb1e0be62bf471307051eaa17000d31d1dfbb970e997dff5 Jul 21 18:06:03.013167 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:03.013124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" event={"ID":"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1","Type":"ContainerStarted","Data":"c9e90a319f8294e4fb1e0be62bf471307051eaa17000d31d1dfbb970e997dff5"} Jul 21 18:06:06.028563 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:06.028526 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" event={"ID":"9fd43684-6823-40bd-9c6f-ebabf4bfb8f1","Type":"ContainerStarted","Data":"4eddc05a59ef5a38f83b35b0c32cfc15ccc9638721ba5463f7816d82b650e380"} Jul 21 18:06:06.049859 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:06:06.049800 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-nwjwg" podStartSLOduration=2.407164781 podStartE2EDuration="5.049782257s" totalImagesPullingTime="2.642617476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:06.048541742 +0000 UTC m=+341.121137442" watchObservedRunningTime="2026-07-21 18:06:06.049782257 +0000 UTC m=+341.122377956" Jul 21 18:07:53.082060 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.081980 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb"] Jul 21 18:07:53.085176 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.085157 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.087670 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.087651 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 21 18:07:53.087800 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.087680 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 18:07:53.087800 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.087712 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 18:07:53.088703 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.088688 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 21 18:07:53.088762 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.088718 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xcwkl\"" Jul 21 18:07:53.094330 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.094309 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb"] Jul 21 18:07:53.153823 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.153792 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/539c1c1c-8444-407c-88f3-5a5c72c95a23-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.154045 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.153834 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g878\" (UniqueName: \"kubernetes.io/projected/539c1c1c-8444-407c-88f3-5a5c72c95a23-kube-api-access-7g878\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.154045 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.153946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/539c1c1c-8444-407c-88f3-5a5c72c95a23-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.255125 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.255079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/539c1c1c-8444-407c-88f3-5a5c72c95a23-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.255314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.255164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/539c1c1c-8444-407c-88f3-5a5c72c95a23-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.255314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.255199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g878\" (UniqueName: \"kubernetes.io/projected/539c1c1c-8444-407c-88f3-5a5c72c95a23-kube-api-access-7g878\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.255855 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.255833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/539c1c1c-8444-407c-88f3-5a5c72c95a23-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.257802 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.257781 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/539c1c1c-8444-407c-88f3-5a5c72c95a23-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.267249 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.267222 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g878\" (UniqueName: \"kubernetes.io/projected/539c1c1c-8444-407c-88f3-5a5c72c95a23-kube-api-access-7g878\") pod \"kuadrant-console-plugin-66b5774985-w7mgb\" (UID: \"539c1c1c-8444-407c-88f3-5a5c72c95a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.395062 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.395025 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" Jul 21 18:07:53.524733 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:53.524705 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb"] Jul 21 18:07:53.527359 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:07:53.527323 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod539c1c1c_8444_407c_88f3_5a5c72c95a23.slice/crio-b1a064f8500bd3c519fa5bac21f2e67babd588ba5f578e5572586f6cc62ca384 WatchSource:0}: Error finding container b1a064f8500bd3c519fa5bac21f2e67babd588ba5f578e5572586f6cc62ca384: Status 404 returned error can't find the container with id b1a064f8500bd3c519fa5bac21f2e67babd588ba5f578e5572586f6cc62ca384 Jul 21 18:07:54.519730 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:07:54.519693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" event={"ID":"539c1c1c-8444-407c-88f3-5a5c72c95a23","Type":"ContainerStarted","Data":"b1a064f8500bd3c519fa5bac21f2e67babd588ba5f578e5572586f6cc62ca384"} Jul 21 18:08:03.100029 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.099985 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg"] Jul 21 18:08:03.121132 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.121098 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg"] Jul 21 18:08:03.121351 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.121251 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.124495 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.124464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v7plk\"" Jul 21 18:08:03.219819 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.219778 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg"] Jul 21 18:08:03.220177 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:08:03.220152 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-mmgcj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" podUID="51c322d6-aaa6-4906-b003-1d4378273247" Jul 21 18:08:03.257771 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.257729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmgcj\" (UniqueName: \"kubernetes.io/projected/51c322d6-aaa6-4906-b003-1d4378273247-kube-api-access-mmgcj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.257968 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.257903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51c322d6-aaa6-4906-b003-1d4378273247-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.359186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.359097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51c322d6-aaa6-4906-b003-1d4378273247-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.359186 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.359145 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmgcj\" (UniqueName: \"kubernetes.io/projected/51c322d6-aaa6-4906-b003-1d4378273247-kube-api-access-mmgcj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.359628 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.359602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51c322d6-aaa6-4906-b003-1d4378273247-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.370009 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.369974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmgcj\" (UniqueName: \"kubernetes.io/projected/51c322d6-aaa6-4906-b003-1d4378273247-kube-api-access-mmgcj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.570420 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.570388 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.583997 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.583970 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:03.728675 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.728636 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg"] Jul 21 18:08:03.734309 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.734277 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg"] Jul 21 18:08:03.763014 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.762975 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51c322d6-aaa6-4906-b003-1d4378273247-extensions-socket-volume\") pod \"51c322d6-aaa6-4906-b003-1d4378273247\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " Jul 21 18:08:03.763211 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.763080 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmgcj\" (UniqueName: \"kubernetes.io/projected/51c322d6-aaa6-4906-b003-1d4378273247-kube-api-access-mmgcj\") pod \"51c322d6-aaa6-4906-b003-1d4378273247\" (UID: \"51c322d6-aaa6-4906-b003-1d4378273247\") " Jul 21 18:08:03.763331 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.763301 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51c322d6-aaa6-4906-b003-1d4378273247-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "51c322d6-aaa6-4906-b003-1d4378273247" (UID: "51c322d6-aaa6-4906-b003-1d4378273247"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:08:03.765697 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.765661 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51c322d6-aaa6-4906-b003-1d4378273247-kube-api-access-mmgcj" (OuterVolumeSpecName: "kube-api-access-mmgcj") pod "51c322d6-aaa6-4906-b003-1d4378273247" (UID: "51c322d6-aaa6-4906-b003-1d4378273247"). InnerVolumeSpecName "kube-api-access-mmgcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:08:03.864061 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.864022 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51c322d6-aaa6-4906-b003-1d4378273247-extensions-socket-volume\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:08:03.864061 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:03.864060 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmgcj\" (UniqueName: \"kubernetes.io/projected/51c322d6-aaa6-4906-b003-1d4378273247-kube-api-access-mmgcj\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:08:04.575195 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:04.575152 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:04.578307 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:08:04.578262 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" is forbidden: User \"system:node:ip-10-0-139-4.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-4.ec2.internal' and this object" podUID="51c322d6-aaa6-4906-b003-1d4378273247" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:04.585954 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:08:04.585914 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" is forbidden: User \"system:node:ip-10-0-139-4.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-4.ec2.internal' and this object" podUID="51c322d6-aaa6-4906-b003-1d4378273247" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:05.532622 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:08:05.532572 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-nz7cg\" is forbidden: User \"system:node:ip-10-0-139-4.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-4.ec2.internal' and this object" podUID="51c322d6-aaa6-4906-b003-1d4378273247" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-nz7cg" Jul 21 18:08:05.534965 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:05.534932 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51c322d6-aaa6-4906-b003-1d4378273247" path="/var/lib/kubelet/pods/51c322d6-aaa6-4906-b003-1d4378273247/volumes" Jul 21 18:08:16.636703 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:16.636662 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" event={"ID":"539c1c1c-8444-407c-88f3-5a5c72c95a23","Type":"ContainerStarted","Data":"21cef5a9d62f619716cb2d9b12f5e0a92021ce04af3eb6394894978c218b4f0a"} Jul 21 18:08:16.657250 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:16.657193 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-w7mgb" podStartSLOduration=0.876624246 podStartE2EDuration="23.657173995s" totalImagesPullingTime="22.780549749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:08:16.655577987 +0000 UTC m=+471.728173687" watchObservedRunningTime="2026-07-21 18:08:16.657173995 +0000 UTC m=+471.729769693" Jul 21 18:08:20.911770 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:20.911729 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8"] Jul 21 18:08:21.003329 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.003289 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8"] Jul 21 18:08:21.003547 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.003419 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.007288 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.007262 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v7plk\"" Jul 21 18:08:21.122177 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.122138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wd7\" (UniqueName: \"kubernetes.io/projected/c09292cd-6306-4aa2-b499-e12bb09f0006-kube-api-access-q7wd7\") pod \"kuadrant-operator-controller-manager-86878c769f-8nsd8\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.122357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.122207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c09292cd-6306-4aa2-b499-e12bb09f0006-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8nsd8\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.223068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.222982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wd7\" (UniqueName: \"kubernetes.io/projected/c09292cd-6306-4aa2-b499-e12bb09f0006-kube-api-access-q7wd7\") pod \"kuadrant-operator-controller-manager-86878c769f-8nsd8\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.223068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.223046 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c09292cd-6306-4aa2-b499-e12bb09f0006-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8nsd8\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.223369 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.223351 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c09292cd-6306-4aa2-b499-e12bb09f0006-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8nsd8\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.233265 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.233229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wd7\" (UniqueName: \"kubernetes.io/projected/c09292cd-6306-4aa2-b499-e12bb09f0006-kube-api-access-q7wd7\") pod \"kuadrant-operator-controller-manager-86878c769f-8nsd8\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.314106 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.314063 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:21.492320 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.492291 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8"] Jul 21 18:08:21.496221 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:08:21.496182 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09292cd_6306_4aa2_b499_e12bb09f0006.slice/crio-98998facc0ea88072dc3ce9c83d058197cdebaa3ead1e86f53f08d95109aef86 WatchSource:0}: Error finding container 98998facc0ea88072dc3ce9c83d058197cdebaa3ead1e86f53f08d95109aef86: Status 404 returned error can't find the container with id 98998facc0ea88072dc3ce9c83d058197cdebaa3ead1e86f53f08d95109aef86 Jul 21 18:08:21.667652 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:21.667617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" event={"ID":"c09292cd-6306-4aa2-b499-e12bb09f0006","Type":"ContainerStarted","Data":"98998facc0ea88072dc3ce9c83d058197cdebaa3ead1e86f53f08d95109aef86"} Jul 21 18:08:27.698122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:27.698084 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" event={"ID":"c09292cd-6306-4aa2-b499-e12bb09f0006","Type":"ContainerStarted","Data":"33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d"} Jul 21 18:08:27.698561 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:27.698292 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:27.719036 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:27.718978 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" podStartSLOduration=1.630947229 podStartE2EDuration="7.718957321s" totalImagesPullingTime="6.088010092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:08:27.716635492 +0000 UTC m=+482.789231191" watchObservedRunningTime="2026-07-21 18:08:27.718957321 +0000 UTC m=+482.791553020" Jul 21 18:08:38.704898 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:38.704859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:08:57.905324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:57.905276 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-xm2gr"] Jul 21 18:08:57.936119 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:57.936085 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xm2gr"] Jul 21 18:08:57.936288 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:57.936180 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:08:57.938965 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:57.938941 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lcp7w\"" Jul 21 18:08:57.960792 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:57.960764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-687zf\" (UniqueName: \"kubernetes.io/projected/5fbbe682-f106-4b72-95db-c1b49688db86-kube-api-access-687zf\") pod \"authorino-75d556bd77-xm2gr\" (UID: \"5fbbe682-f106-4b72-95db-c1b49688db86\") " pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:08:58.061874 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:58.061836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-687zf\" (UniqueName: \"kubernetes.io/projected/5fbbe682-f106-4b72-95db-c1b49688db86-kube-api-access-687zf\") pod \"authorino-75d556bd77-xm2gr\" (UID: \"5fbbe682-f106-4b72-95db-c1b49688db86\") " pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:08:58.071969 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:58.071939 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-687zf\" (UniqueName: \"kubernetes.io/projected/5fbbe682-f106-4b72-95db-c1b49688db86-kube-api-access-687zf\") pod \"authorino-75d556bd77-xm2gr\" (UID: \"5fbbe682-f106-4b72-95db-c1b49688db86\") " pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:08:58.245730 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:58.245628 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:08:58.386519 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:58.386365 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xm2gr"] Jul 21 18:08:58.389014 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:08:58.388978 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fbbe682_f106_4b72_95db_c1b49688db86.slice/crio-2b18727affe60c8104559f5ce816b2fbe59821b0a0ebe6d0d801883978debab3 WatchSource:0}: Error finding container 2b18727affe60c8104559f5ce816b2fbe59821b0a0ebe6d0d801883978debab3: Status 404 returned error can't find the container with id 2b18727affe60c8104559f5ce816b2fbe59821b0a0ebe6d0d801883978debab3 Jul 21 18:08:58.853656 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:08:58.853619 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xm2gr" event={"ID":"5fbbe682-f106-4b72-95db-c1b49688db86","Type":"ContainerStarted","Data":"2b18727affe60c8104559f5ce816b2fbe59821b0a0ebe6d0d801883978debab3"} Jul 21 18:09:02.877853 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:02.877813 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xm2gr" event={"ID":"5fbbe682-f106-4b72-95db-c1b49688db86","Type":"ContainerStarted","Data":"58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed"} Jul 21 18:09:02.894190 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:02.894134 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-xm2gr" podStartSLOduration=2.339530672 podStartE2EDuration="5.894116598s" totalImagesPullingTime="3.554585926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:02.893599112 +0000 UTC m=+517.966194811" watchObservedRunningTime="2026-07-21 18:09:02.894116598 +0000 UTC m=+517.966712296" Jul 21 18:09:30.360594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:30.360556 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xm2gr"] Jul 21 18:09:30.361143 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:30.360741 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-xm2gr" podUID="5fbbe682-f106-4b72-95db-c1b49688db86" containerName="authorino" containerID="cri-o://58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed" gracePeriod=30 Jul 21 18:09:30.611043 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:30.610969 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:09:30.774169 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:30.774137 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-687zf\" (UniqueName: \"kubernetes.io/projected/5fbbe682-f106-4b72-95db-c1b49688db86-kube-api-access-687zf\") pod \"5fbbe682-f106-4b72-95db-c1b49688db86\" (UID: \"5fbbe682-f106-4b72-95db-c1b49688db86\") " Jul 21 18:09:30.776561 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:30.776524 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fbbe682-f106-4b72-95db-c1b49688db86-kube-api-access-687zf" (OuterVolumeSpecName: "kube-api-access-687zf") pod "5fbbe682-f106-4b72-95db-c1b49688db86" (UID: "5fbbe682-f106-4b72-95db-c1b49688db86"). InnerVolumeSpecName "kube-api-access-687zf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:09:30.875929 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:30.875901 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-687zf\" (UniqueName: \"kubernetes.io/projected/5fbbe682-f106-4b72-95db-c1b49688db86-kube-api-access-687zf\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:09:31.007242 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.007203 2560 generic.go:356] "Generic (PLEG): container finished" podID="5fbbe682-f106-4b72-95db-c1b49688db86" containerID="58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed" exitCode=0 Jul 21 18:09:31.007417 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.007247 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xm2gr" Jul 21 18:09:31.007417 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.007249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xm2gr" event={"ID":"5fbbe682-f106-4b72-95db-c1b49688db86","Type":"ContainerDied","Data":"58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed"} Jul 21 18:09:31.007417 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.007372 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xm2gr" event={"ID":"5fbbe682-f106-4b72-95db-c1b49688db86","Type":"ContainerDied","Data":"2b18727affe60c8104559f5ce816b2fbe59821b0a0ebe6d0d801883978debab3"} Jul 21 18:09:31.007417 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.007398 2560 scope.go:117] "RemoveContainer" containerID="58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed" Jul 21 18:09:31.017122 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.017098 2560 scope.go:117] "RemoveContainer" containerID="58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed" Jul 21 18:09:31.017478 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:09:31.017434 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed\": container with ID starting with 58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed not found: ID does not exist" containerID="58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed" Jul 21 18:09:31.017577 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.017489 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed"} err="failed to get container status \"58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed\": rpc error: code = NotFound desc = could not find container \"58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed\": container with ID starting with 58f970418f01dba67b47d700cc99810fed06f44de0691f587327279fcb2ec5ed not found: ID does not exist" Jul 21 18:09:31.036691 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.036656 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xm2gr"] Jul 21 18:09:31.049553 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.049515 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xm2gr"] Jul 21 18:09:31.532566 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:09:31.532533 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fbbe682-f106-4b72-95db-c1b49688db86" path="/var/lib/kubelet/pods/5fbbe682-f106-4b72-95db-c1b49688db86/volumes" Jul 21 18:10:11.112880 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.112841 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gcnpz"] Jul 21 18:10:11.115378 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.113821 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fbbe682-f106-4b72-95db-c1b49688db86" containerName="authorino" Jul 21 18:10:11.117707 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.113857 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fbbe682-f106-4b72-95db-c1b49688db86" containerName="authorino" Jul 21 18:10:11.117707 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.115884 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fbbe682-f106-4b72-95db-c1b49688db86" containerName="authorino" Jul 21 18:10:11.120314 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.120016 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.123046 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.123020 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 21 18:10:11.124171 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.124151 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-s54wd"] Jul 21 18:10:11.127601 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.127579 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.129057 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.129032 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gcnpz"] Jul 21 18:10:11.130993 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.130973 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-88kt8\"" Jul 21 18:10:11.138626 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.138600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-s54wd"] Jul 21 18:10:11.239489 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.239427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0347d7e9-6450-45cb-b327-8dd62c0324ca-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gcnpz\" (UID: \"0347d7e9-6450-45cb-b327-8dd62c0324ca\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.239671 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.239514 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv8gs\" (UniqueName: \"kubernetes.io/projected/959c0599-2959-40ce-aaca-539d3681d648-kube-api-access-vv8gs\") pod \"payload-processing-5bf47cd988-s54wd\" (UID: \"959c0599-2959-40ce-aaca-539d3681d648\") " pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.239671 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.239623 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959c0599-2959-40ce-aaca-539d3681d648-plugins-config-volume\") pod \"payload-processing-5bf47cd988-s54wd\" (UID: \"959c0599-2959-40ce-aaca-539d3681d648\") " pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.239778 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.239680 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96s6q\" (UniqueName: \"kubernetes.io/projected/0347d7e9-6450-45cb-b327-8dd62c0324ca-kube-api-access-96s6q\") pod \"payload-pre-processing-7677b774bb-gcnpz\" (UID: \"0347d7e9-6450-45cb-b327-8dd62c0324ca\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.340697 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.340663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0347d7e9-6450-45cb-b327-8dd62c0324ca-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gcnpz\" (UID: \"0347d7e9-6450-45cb-b327-8dd62c0324ca\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.340888 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.340708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv8gs\" (UniqueName: \"kubernetes.io/projected/959c0599-2959-40ce-aaca-539d3681d648-kube-api-access-vv8gs\") pod \"payload-processing-5bf47cd988-s54wd\" (UID: \"959c0599-2959-40ce-aaca-539d3681d648\") " pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.340888 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.340766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959c0599-2959-40ce-aaca-539d3681d648-plugins-config-volume\") pod \"payload-processing-5bf47cd988-s54wd\" (UID: \"959c0599-2959-40ce-aaca-539d3681d648\") " pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.340888 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.340796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96s6q\" (UniqueName: \"kubernetes.io/projected/0347d7e9-6450-45cb-b327-8dd62c0324ca-kube-api-access-96s6q\") pod \"payload-pre-processing-7677b774bb-gcnpz\" (UID: \"0347d7e9-6450-45cb-b327-8dd62c0324ca\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.341620 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.341596 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0347d7e9-6450-45cb-b327-8dd62c0324ca-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-gcnpz\" (UID: \"0347d7e9-6450-45cb-b327-8dd62c0324ca\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.341690 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.341596 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959c0599-2959-40ce-aaca-539d3681d648-plugins-config-volume\") pod \"payload-processing-5bf47cd988-s54wd\" (UID: \"959c0599-2959-40ce-aaca-539d3681d648\") " pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.349538 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.349512 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv8gs\" (UniqueName: \"kubernetes.io/projected/959c0599-2959-40ce-aaca-539d3681d648-kube-api-access-vv8gs\") pod \"payload-processing-5bf47cd988-s54wd\" (UID: \"959c0599-2959-40ce-aaca-539d3681d648\") " pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.349652 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.349584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96s6q\" (UniqueName: \"kubernetes.io/projected/0347d7e9-6450-45cb-b327-8dd62c0324ca-kube-api-access-96s6q\") pod \"payload-pre-processing-7677b774bb-gcnpz\" (UID: \"0347d7e9-6450-45cb-b327-8dd62c0324ca\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.434125 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.434085 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:11.441323 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.441298 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:11.583262 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.583236 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-gcnpz"] Jul 21 18:10:11.586613 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:10:11.586581 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0347d7e9_6450_45cb_b327_8dd62c0324ca.slice/crio-a942c9b83699bcdca8e589a5f83dad1225e97ff6d00bb70cf9b4572bcff2aae6 WatchSource:0}: Error finding container a942c9b83699bcdca8e589a5f83dad1225e97ff6d00bb70cf9b4572bcff2aae6: Status 404 returned error can't find the container with id a942c9b83699bcdca8e589a5f83dad1225e97ff6d00bb70cf9b4572bcff2aae6 Jul 21 18:10:11.606345 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:11.606324 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-s54wd"] Jul 21 18:10:11.608598 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:10:11.608573 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959c0599_2959_40ce_aaca_539d3681d648.slice/crio-e26a25b3916bd5b1aa9576dcaaff1b822ec8d9a4b7018d908ff73f4ead3497b4 WatchSource:0}: Error finding container e26a25b3916bd5b1aa9576dcaaff1b822ec8d9a4b7018d908ff73f4ead3497b4: Status 404 returned error can't find the container with id e26a25b3916bd5b1aa9576dcaaff1b822ec8d9a4b7018d908ff73f4ead3497b4 Jul 21 18:10:12.205401 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:12.205364 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" event={"ID":"959c0599-2959-40ce-aaca-539d3681d648","Type":"ContainerStarted","Data":"e26a25b3916bd5b1aa9576dcaaff1b822ec8d9a4b7018d908ff73f4ead3497b4"} Jul 21 18:10:12.206431 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:12.206407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" event={"ID":"0347d7e9-6450-45cb-b327-8dd62c0324ca","Type":"ContainerStarted","Data":"a942c9b83699bcdca8e589a5f83dad1225e97ff6d00bb70cf9b4572bcff2aae6"} Jul 21 18:10:14.219282 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:14.219246 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" event={"ID":"959c0599-2959-40ce-aaca-539d3681d648","Type":"ContainerStarted","Data":"3d57c20de893dba459a150f79efe6dc0e22c7e7e55634bb985ae4e8dbb028dde"} Jul 21 18:10:14.219807 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:14.219315 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:14.220896 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:14.220863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" event={"ID":"0347d7e9-6450-45cb-b327-8dd62c0324ca","Type":"ContainerStarted","Data":"26dc82c9cf271b55d5d733b4d9ae16f2934cad2855ffac9fcf7c5b38f6b209e4"} Jul 21 18:10:14.221025 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:14.220916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:10:14.237095 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:14.237037 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" podStartSLOduration=1.31039392 podStartE2EDuration="3.237021652s" totalImagesPullingTime="1.926627732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:10:14.234704711 +0000 UTC m=+589.307300409" watchObservedRunningTime="2026-07-21 18:10:14.237021652 +0000 UTC m=+589.309617349" Jul 21 18:10:14.251127 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:14.251064 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" podStartSLOduration=1.309914646 podStartE2EDuration="3.251046615s" totalImagesPullingTime="1.941131969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:10:14.24963741 +0000 UTC m=+589.322233122" watchObservedRunningTime="2026-07-21 18:10:14.251046615 +0000 UTC m=+589.323642331" Jul 21 18:10:25.227966 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:25.227906 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" podUID="0347d7e9-6450-45cb-b327-8dd62c0324ca" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.29:9004: connect: connection refused" Jul 21 18:10:25.228351 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:25.228303 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-s54wd" Jul 21 18:10:25.679041 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:25.678927 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:10:25.692966 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:25.681263 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:10:35.226975 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:10:35.226943 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-gcnpz" Jul 21 18:11:06.246837 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.246757 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4"] Jul 21 18:11:06.255102 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.255072 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.259200 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.259171 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 18:11:06.259200 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.259171 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g5lsg\"" Jul 21 18:11:06.259391 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.259201 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 18:11:06.259391 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.259171 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 21 18:11:06.263136 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.263111 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4"] Jul 21 18:11:06.352058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.352025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.352225 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.352071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhvb2\" (UniqueName: \"kubernetes.io/projected/99e68066-b2a9-441b-8dfe-05c2ea89cc02-kube-api-access-nhvb2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.352225 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.352097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.352225 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.352140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.352225 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.352182 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e68066-b2a9-441b-8dfe-05c2ea89cc02-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.352356 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.352224 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453433 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453649 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhvb2\" (UniqueName: \"kubernetes.io/projected/99e68066-b2a9-441b-8dfe-05c2ea89cc02-kube-api-access-nhvb2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453649 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453649 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453555 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453749 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e68066-b2a9-441b-8dfe-05c2ea89cc02-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453794 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453905 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453887 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453957 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.453991 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.453969 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.456093 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.456067 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e68066-b2a9-441b-8dfe-05c2ea89cc02-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.456596 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.456579 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e68066-b2a9-441b-8dfe-05c2ea89cc02-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.462958 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.462929 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhvb2\" (UniqueName: \"kubernetes.io/projected/99e68066-b2a9-441b-8dfe-05c2ea89cc02-kube-api-access-nhvb2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9nj4\" (UID: \"99e68066-b2a9-441b-8dfe-05c2ea89cc02\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.566812 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.566721 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:06.739694 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.739665 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4"] Jul 21 18:11:06.741795 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:11:06.741764 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e68066_b2a9_441b_8dfe_05c2ea89cc02.slice/crio-2db48369a4e04ea94e30b0b6f0cd49c475f90798e486d4e93573a095eaf96d56 WatchSource:0}: Error finding container 2db48369a4e04ea94e30b0b6f0cd49c475f90798e486d4e93573a095eaf96d56: Status 404 returned error can't find the container with id 2db48369a4e04ea94e30b0b6f0cd49c475f90798e486d4e93573a095eaf96d56 Jul 21 18:11:06.743097 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:06.743082 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:11:07.479454 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:07.479418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" event={"ID":"99e68066-b2a9-441b-8dfe-05c2ea89cc02","Type":"ContainerStarted","Data":"2db48369a4e04ea94e30b0b6f0cd49c475f90798e486d4e93573a095eaf96d56"} Jul 21 18:11:09.492077 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:09.492042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" event={"ID":"99e68066-b2a9-441b-8dfe-05c2ea89cc02","Type":"ContainerStarted","Data":"2800576b32c6b9875b9140e189a32e82519e537dfe54692d81879ac66f31bb61"} Jul 21 18:11:16.567298 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:16.567255 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:16.567834 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:16.567336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:16.579836 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:16.579811 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:16.604761 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:16.604705 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" podStartSLOduration=8.769819247000001 podStartE2EDuration="10.604687218s" totalImagesPullingTime="1.834867971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:11:09.516482867 +0000 UTC m=+644.589078566" watchObservedRunningTime="2026-07-21 18:11:16.604687218 +0000 UTC m=+651.677282949" Jul 21 18:11:17.541656 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:17.541626 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9nj4" Jul 21 18:11:34.849788 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:34.849746 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n"] Jul 21 18:11:34.856092 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:34.856066 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:34.858668 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:34.858635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 18:11:34.868485 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:34.868434 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n"] Jul 21 18:11:35.029340 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.029301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.029564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.029363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303d3d5c-78d8-496d-ad22-3e1704c56892-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.029564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.029424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.029564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.029465 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.029564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.029488 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.029564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.029523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbf8m\" (UniqueName: \"kubernetes.io/projected/303d3d5c-78d8-496d-ad22-3e1704c56892-kube-api-access-fbf8m\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.129988 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.129961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303d3d5c-78d8-496d-ad22-3e1704c56892-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130129 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130129 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130129 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130048 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130129 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbf8m\" (UniqueName: \"kubernetes.io/projected/303d3d5c-78d8-496d-ad22-3e1704c56892-kube-api-access-fbf8m\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130320 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130492 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130588 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130538 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.130627 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.130615 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.132277 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.132258 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/303d3d5c-78d8-496d-ad22-3e1704c56892-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.132564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.132547 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/303d3d5c-78d8-496d-ad22-3e1704c56892-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.139625 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.139602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbf8m\" (UniqueName: \"kubernetes.io/projected/303d3d5c-78d8-496d-ad22-3e1704c56892-kube-api-access-fbf8m\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n\" (UID: \"303d3d5c-78d8-496d-ad22-3e1704c56892\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.165440 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.165419 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:35.302006 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.301844 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n"] Jul 21 18:11:35.304965 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:11:35.304937 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod303d3d5c_78d8_496d_ad22_3e1704c56892.slice/crio-aead0c5153fa40ee6669ee4b53a8c4c64d8ec2e6be9bee0c1bb572d709c326dc WatchSource:0}: Error finding container aead0c5153fa40ee6669ee4b53a8c4c64d8ec2e6be9bee0c1bb572d709c326dc: Status 404 returned error can't find the container with id aead0c5153fa40ee6669ee4b53a8c4c64d8ec2e6be9bee0c1bb572d709c326dc Jul 21 18:11:35.617125 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.617098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" event={"ID":"303d3d5c-78d8-496d-ad22-3e1704c56892","Type":"ContainerStarted","Data":"88b3a2d92e2a62c783f2ea025d2ac601f5e60d66deb55add345b5422172cb8b9"} Jul 21 18:11:35.617260 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.617136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" event={"ID":"303d3d5c-78d8-496d-ad22-3e1704c56892","Type":"ContainerStarted","Data":"aead0c5153fa40ee6669ee4b53a8c4c64d8ec2e6be9bee0c1bb572d709c326dc"} Jul 21 18:11:35.636329 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:35.636283 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" podStartSLOduration=1.396810895 podStartE2EDuration="1.636267252s" totalImagesPullingTime="239.456357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:11:35.634770099 +0000 UTC m=+670.707365796" watchObservedRunningTime="2026-07-21 18:11:35.636267252 +0000 UTC m=+670.708862948" Jul 21 18:11:45.166364 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:45.166324 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:45.166888 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:45.166427 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:45.183267 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:45.183235 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:11:45.674157 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:11:45.674126 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n" Jul 21 18:12:30.133004 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.132962 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-744f66d67-27qz2"] Jul 21 18:12:30.136518 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.136493 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.139303 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.139285 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lcp7w\"" Jul 21 18:12:30.140218 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.140197 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 18:12:30.146037 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.146006 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-744f66d67-27qz2"] Jul 21 18:12:30.255276 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.255241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsrh\" (UniqueName: \"kubernetes.io/projected/0a40bac6-ffd7-46d1-9a63-20b0468d63e7-kube-api-access-zlsrh\") pod \"authorino-744f66d67-27qz2\" (UID: \"0a40bac6-ffd7-46d1-9a63-20b0468d63e7\") " pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.255495 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.255299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a40bac6-ffd7-46d1-9a63-20b0468d63e7-tls-cert\") pod \"authorino-744f66d67-27qz2\" (UID: \"0a40bac6-ffd7-46d1-9a63-20b0468d63e7\") " pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.356258 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.356222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsrh\" (UniqueName: \"kubernetes.io/projected/0a40bac6-ffd7-46d1-9a63-20b0468d63e7-kube-api-access-zlsrh\") pod \"authorino-744f66d67-27qz2\" (UID: \"0a40bac6-ffd7-46d1-9a63-20b0468d63e7\") " pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.356503 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.356302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a40bac6-ffd7-46d1-9a63-20b0468d63e7-tls-cert\") pod \"authorino-744f66d67-27qz2\" (UID: \"0a40bac6-ffd7-46d1-9a63-20b0468d63e7\") " pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.359021 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.358998 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a40bac6-ffd7-46d1-9a63-20b0468d63e7-tls-cert\") pod \"authorino-744f66d67-27qz2\" (UID: \"0a40bac6-ffd7-46d1-9a63-20b0468d63e7\") " pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.365820 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.365785 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsrh\" (UniqueName: \"kubernetes.io/projected/0a40bac6-ffd7-46d1-9a63-20b0468d63e7-kube-api-access-zlsrh\") pod \"authorino-744f66d67-27qz2\" (UID: \"0a40bac6-ffd7-46d1-9a63-20b0468d63e7\") " pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.446503 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.446410 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-744f66d67-27qz2" Jul 21 18:12:30.600133 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.600105 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-744f66d67-27qz2"] Jul 21 18:12:30.883914 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:30.883880 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-744f66d67-27qz2" event={"ID":"0a40bac6-ffd7-46d1-9a63-20b0468d63e7","Type":"ContainerStarted","Data":"59a0173be52ad24ae0471953b5ed9be2232a754400a8ba1a57bedb404e44a0f5"} Jul 21 18:12:31.897834 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:31.897790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-744f66d67-27qz2" event={"ID":"0a40bac6-ffd7-46d1-9a63-20b0468d63e7","Type":"ContainerStarted","Data":"435273d3274c97fe5aa99a24267dc7640598c556c6c1920dc34c9d7fadb00413"} Jul 21 18:12:31.918657 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:12:31.918595 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-744f66d67-27qz2" podStartSLOduration=1.472720321 podStartE2EDuration="1.918571806s" totalImagesPullingTime="445.851485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:12:31.918214346 +0000 UTC m=+726.990810046" watchObservedRunningTime="2026-07-21 18:12:31.918571806 +0000 UTC m=+726.991167507" Jul 21 18:15:25.710481 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:15:25.710352 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:15:25.716674 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:15:25.712733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:19:27.652530 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:27.652486 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8"] Jul 21 18:19:27.653195 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:27.652774 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" podUID="c09292cd-6306-4aa2-b499-e12bb09f0006" containerName="manager" containerID="cri-o://33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d" gracePeriod=10 Jul 21 18:19:31.110990 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.110965 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:19:31.196739 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.196651 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7wd7\" (UniqueName: \"kubernetes.io/projected/c09292cd-6306-4aa2-b499-e12bb09f0006-kube-api-access-q7wd7\") pod \"c09292cd-6306-4aa2-b499-e12bb09f0006\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " Jul 21 18:19:31.196897 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.196738 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c09292cd-6306-4aa2-b499-e12bb09f0006-extensions-socket-volume\") pod \"c09292cd-6306-4aa2-b499-e12bb09f0006\" (UID: \"c09292cd-6306-4aa2-b499-e12bb09f0006\") " Jul 21 18:19:31.197055 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.197030 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c09292cd-6306-4aa2-b499-e12bb09f0006-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c09292cd-6306-4aa2-b499-e12bb09f0006" (UID: "c09292cd-6306-4aa2-b499-e12bb09f0006"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:19:31.198834 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.198807 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c09292cd-6306-4aa2-b499-e12bb09f0006-kube-api-access-q7wd7" (OuterVolumeSpecName: "kube-api-access-q7wd7") pod "c09292cd-6306-4aa2-b499-e12bb09f0006" (UID: "c09292cd-6306-4aa2-b499-e12bb09f0006"). InnerVolumeSpecName "kube-api-access-q7wd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:19:31.297854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.297820 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7wd7\" (UniqueName: \"kubernetes.io/projected/c09292cd-6306-4aa2-b499-e12bb09f0006-kube-api-access-q7wd7\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:19:31.297854 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.297848 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c09292cd-6306-4aa2-b499-e12bb09f0006-extensions-socket-volume\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:19:31.972649 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.972603 2560 generic.go:356] "Generic (PLEG): container finished" podID="c09292cd-6306-4aa2-b499-e12bb09f0006" containerID="33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d" exitCode=0 Jul 21 18:19:31.972842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.972658 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" Jul 21 18:19:31.972842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.972671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" event={"ID":"c09292cd-6306-4aa2-b499-e12bb09f0006","Type":"ContainerDied","Data":"33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d"} Jul 21 18:19:31.972842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.972716 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8" event={"ID":"c09292cd-6306-4aa2-b499-e12bb09f0006","Type":"ContainerDied","Data":"98998facc0ea88072dc3ce9c83d058197cdebaa3ead1e86f53f08d95109aef86"} Jul 21 18:19:31.972842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.972745 2560 scope.go:117] "RemoveContainer" containerID="33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d" Jul 21 18:19:31.983164 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.983141 2560 scope.go:117] "RemoveContainer" containerID="33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d" Jul 21 18:19:31.983430 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:19:31.983405 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d\": container with ID starting with 33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d not found: ID does not exist" containerID="33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d" Jul 21 18:19:31.983562 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.983437 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d"} err="failed to get container status \"33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d\": rpc error: code = NotFound desc = could not find container \"33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d\": container with ID starting with 33a2191b412f6dbcd82d5e7205c157c15bb1c74f0560f2f25b28f7d88708a09d not found: ID does not exist" Jul 21 18:19:31.994602 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.994572 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8"] Jul 21 18:19:31.998959 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:31.998933 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8nsd8"] Jul 21 18:19:33.533074 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:19:33.533041 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c09292cd-6306-4aa2-b499-e12bb09f0006" path="/var/lib/kubelet/pods/c09292cd-6306-4aa2-b499-e12bb09f0006/volumes" Jul 21 18:20:25.742803 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:25.742654 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:20:25.747108 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:25.745480 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:20:28.812867 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.812833 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk"] Jul 21 18:20:28.813245 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.813233 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c09292cd-6306-4aa2-b499-e12bb09f0006" containerName="manager" Jul 21 18:20:28.813287 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.813247 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c09292cd-6306-4aa2-b499-e12bb09f0006" containerName="manager" Jul 21 18:20:28.813332 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.813323 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c09292cd-6306-4aa2-b499-e12bb09f0006" containerName="manager" Jul 21 18:20:28.816337 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.816320 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:28.818892 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.818875 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v7plk\"" Jul 21 18:20:28.831356 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.831179 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk"] Jul 21 18:20:28.948585 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.948551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q7vxk\" (UID: \"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:28.948745 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:28.948596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4r8j\" (UniqueName: \"kubernetes.io/projected/5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc-kube-api-access-r4r8j\") pod \"kuadrant-operator-controller-manager-86878c769f-q7vxk\" (UID: \"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:29.049904 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:29.049859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q7vxk\" (UID: \"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:29.050091 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:29.049923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4r8j\" (UniqueName: \"kubernetes.io/projected/5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc-kube-api-access-r4r8j\") pod \"kuadrant-operator-controller-manager-86878c769f-q7vxk\" (UID: \"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:29.050251 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:29.050232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q7vxk\" (UID: \"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:29.068713 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:29.068655 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4r8j\" (UniqueName: \"kubernetes.io/projected/5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc-kube-api-access-r4r8j\") pod \"kuadrant-operator-controller-manager-86878c769f-q7vxk\" (UID: \"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:29.126749 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:29.126713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:29.273775 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:29.273747 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk"] Jul 21 18:20:29.276682 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:20:29.276655 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd833e8_77c1_4f1f_95de_2b86fe9ca4cc.slice/crio-007f82104635f95803d0eaa78e3b292c003314c07cd79828981044c65fb9057a WatchSource:0}: Error finding container 007f82104635f95803d0eaa78e3b292c003314c07cd79828981044c65fb9057a: Status 404 returned error can't find the container with id 007f82104635f95803d0eaa78e3b292c003314c07cd79828981044c65fb9057a Jul 21 18:20:30.253606 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:30.253568 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" event={"ID":"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc","Type":"ContainerStarted","Data":"ba8ea8af472cd021ad7b790e2055f703d88aef80c794da80e3791bf5baf26ddf"} Jul 21 18:20:30.253963 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:30.253620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" event={"ID":"5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc","Type":"ContainerStarted","Data":"007f82104635f95803d0eaa78e3b292c003314c07cd79828981044c65fb9057a"} Jul 21 18:20:30.253963 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:30.253690 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:20:30.273916 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:30.273868 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" podStartSLOduration=2.273852299 podStartE2EDuration="2.273852299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:20:30.271341202 +0000 UTC m=+1205.343936910" watchObservedRunningTime="2026-07-21 18:20:30.273852299 +0000 UTC m=+1205.346447996" Jul 21 18:20:41.260007 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:20:41.259974 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q7vxk" Jul 21 18:25:25.777481 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:25:25.777350 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:25:25.781428 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:25:25.780815 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:30:25.807876 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:30:25.807760 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:30:25.814440 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:30:25.812369 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:35:25.847001 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:35:25.846881 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:35:25.850223 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:35:25.849711 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:36:58.640235 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.640197 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn"] Jul 21 18:36:58.643828 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.643806 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.647271 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.647245 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b2c46d\"/\"openshift-service-ca.crt\"" Jul 21 18:36:58.647408 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.647245 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b2c46d\"/\"default-dockercfg-5m4mq\"" Jul 21 18:36:58.647408 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.647305 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b2c46d\"/\"kube-root-ca.crt\"" Jul 21 18:36:58.647408 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.647332 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b2c46d\"/\"test-model-b2c46d-kserve-self-signed-certs\"" Jul 21 18:36:58.654221 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.654197 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn"] Jul 21 18:36:58.735167 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.735136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c263f0-5102-4f0e-b30b-a797518d5093-tls-certs\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.735362 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.735202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-model-cache\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.735362 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.735284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-tmp-dir\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.735362 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.735329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-home\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.735362 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.735345 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-dshm\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.735556 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.735385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms8dd\" (UniqueName: \"kubernetes.io/projected/06c263f0-5102-4f0e-b30b-a797518d5093-kube-api-access-ms8dd\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.836576 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.836541 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c263f0-5102-4f0e-b30b-a797518d5093-tls-certs\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.836758 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.836622 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-model-cache\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.836758 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.836658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-tmp-dir\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.836758 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.836689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-home\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.836758 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.836703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-dshm\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.836758 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.836731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms8dd\" (UniqueName: \"kubernetes.io/projected/06c263f0-5102-4f0e-b30b-a797518d5093-kube-api-access-ms8dd\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.837137 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.837105 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-model-cache\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.837246 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.837114 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-tmp-dir\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.837246 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.837198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-home\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.838951 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.838931 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-dshm\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.839224 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.839208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c263f0-5102-4f0e-b30b-a797518d5093-tls-certs\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.845614 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.845588 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms8dd\" (UniqueName: \"kubernetes.io/projected/06c263f0-5102-4f0e-b30b-a797518d5093-kube-api-access-ms8dd\") pod \"test-model-b2c46d-kserve-fc9b9d48c-6kzmn\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:58.952855 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:58.952775 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:36:59.087637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:36:59.087614 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn"] Jul 21 18:36:59.090039 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:36:59.090013 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06c263f0_5102_4f0e_b30b_a797518d5093.slice/crio-378cb5254295dd66b17bdae280dd141a294eb0904fbb207626a29a3091d35377 WatchSource:0}: Error finding container 378cb5254295dd66b17bdae280dd141a294eb0904fbb207626a29a3091d35377: Status 404 returned error can't find the container with id 378cb5254295dd66b17bdae280dd141a294eb0904fbb207626a29a3091d35377 Jul 21 18:37:00.061063 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:00.061031 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" event={"ID":"06c263f0-5102-4f0e-b30b-a797518d5093","Type":"ContainerStarted","Data":"32bc8ea483b972059668998b7413eb5a834e53ba966c26488aa64bf79abb9c59"} Jul 21 18:37:00.061430 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:00.061073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" event={"ID":"06c263f0-5102-4f0e-b30b-a797518d5093","Type":"ContainerStarted","Data":"378cb5254295dd66b17bdae280dd141a294eb0904fbb207626a29a3091d35377"} Jul 21 18:37:00.080125 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:00.080070 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" podStartSLOduration=2.080052533 podStartE2EDuration="2.080052533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:37:00.078760904 +0000 UTC m=+2195.151356599" watchObservedRunningTime="2026-07-21 18:37:00.080052533 +0000 UTC m=+2195.152648229" Jul 21 18:37:08.953733 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:08.953690 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:37:08.954118 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:08.953746 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:37:08.966047 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:08.966020 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:37:09.122645 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:09.122610 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:37:48.190251 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:37:48.190159 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn"] Jul 21 18:38:03.210712 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.210674 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" podUID="06c263f0-5102-4f0e-b30b-a797518d5093" containerName="main" containerID="cri-o://32bc8ea483b972059668998b7413eb5a834e53ba966c26488aa64bf79abb9c59" gracePeriod=45 Jul 21 18:38:03.384593 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.384566 2560 generic.go:356] "Generic (PLEG): container finished" podID="06c263f0-5102-4f0e-b30b-a797518d5093" containerID="32bc8ea483b972059668998b7413eb5a834e53ba966c26488aa64bf79abb9c59" exitCode=0 Jul 21 18:38:03.384726 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.384657 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" event={"ID":"06c263f0-5102-4f0e-b30b-a797518d5093","Type":"ContainerDied","Data":"32bc8ea483b972059668998b7413eb5a834e53ba966c26488aa64bf79abb9c59"} Jul 21 18:38:03.461200 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.461139 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:38:03.535694 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535668 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c263f0-5102-4f0e-b30b-a797518d5093-tls-certs\") pod \"06c263f0-5102-4f0e-b30b-a797518d5093\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " Jul 21 18:38:03.535872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535702 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-tmp-dir\") pod \"06c263f0-5102-4f0e-b30b-a797518d5093\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " Jul 21 18:38:03.535872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535728 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-dshm\") pod \"06c263f0-5102-4f0e-b30b-a797518d5093\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " Jul 21 18:38:03.535872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535771 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-model-cache\") pod \"06c263f0-5102-4f0e-b30b-a797518d5093\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " Jul 21 18:38:03.535872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535804 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms8dd\" (UniqueName: \"kubernetes.io/projected/06c263f0-5102-4f0e-b30b-a797518d5093-kube-api-access-ms8dd\") pod \"06c263f0-5102-4f0e-b30b-a797518d5093\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " Jul 21 18:38:03.535872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535869 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-home\") pod \"06c263f0-5102-4f0e-b30b-a797518d5093\" (UID: \"06c263f0-5102-4f0e-b30b-a797518d5093\") " Jul 21 18:38:03.536108 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.535935 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06c263f0-5102-4f0e-b30b-a797518d5093" (UID: "06c263f0-5102-4f0e-b30b-a797518d5093"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:38:03.536108 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.536027 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-model-cache" (OuterVolumeSpecName: "model-cache") pod "06c263f0-5102-4f0e-b30b-a797518d5093" (UID: "06c263f0-5102-4f0e-b30b-a797518d5093"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:38:03.536256 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.536234 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-home" (OuterVolumeSpecName: "home") pod "06c263f0-5102-4f0e-b30b-a797518d5093" (UID: "06c263f0-5102-4f0e-b30b-a797518d5093"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:38:03.536256 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.536245 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-tmp-dir\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:38:03.536626 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.536267 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-model-cache\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:38:03.538113 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.538093 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06c263f0-5102-4f0e-b30b-a797518d5093-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06c263f0-5102-4f0e-b30b-a797518d5093" (UID: "06c263f0-5102-4f0e-b30b-a797518d5093"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:38:03.538113 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.538101 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-dshm" (OuterVolumeSpecName: "dshm") pod "06c263f0-5102-4f0e-b30b-a797518d5093" (UID: "06c263f0-5102-4f0e-b30b-a797518d5093"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:38:03.538232 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.538118 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c263f0-5102-4f0e-b30b-a797518d5093-kube-api-access-ms8dd" (OuterVolumeSpecName: "kube-api-access-ms8dd") pod "06c263f0-5102-4f0e-b30b-a797518d5093" (UID: "06c263f0-5102-4f0e-b30b-a797518d5093"). InnerVolumeSpecName "kube-api-access-ms8dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:38:03.637425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.637391 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-home\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:38:03.637425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.637418 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c263f0-5102-4f0e-b30b-a797518d5093-tls-certs\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:38:03.637425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.637427 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c263f0-5102-4f0e-b30b-a797518d5093-dshm\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:38:03.637425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:03.637435 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms8dd\" (UniqueName: \"kubernetes.io/projected/06c263f0-5102-4f0e-b30b-a797518d5093-kube-api-access-ms8dd\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:38:04.390804 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:04.390768 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" Jul 21 18:38:04.391284 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:04.390804 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn" event={"ID":"06c263f0-5102-4f0e-b30b-a797518d5093","Type":"ContainerDied","Data":"378cb5254295dd66b17bdae280dd141a294eb0904fbb207626a29a3091d35377"} Jul 21 18:38:04.391284 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:04.390856 2560 scope.go:117] "RemoveContainer" containerID="32bc8ea483b972059668998b7413eb5a834e53ba966c26488aa64bf79abb9c59" Jul 21 18:38:04.415048 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:04.415015 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn"] Jul 21 18:38:04.420881 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:04.420848 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b2c46d/test-model-b2c46d-kserve-fc9b9d48c-6kzmn"] Jul 21 18:38:05.533675 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:38:05.533639 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06c263f0-5102-4f0e-b30b-a797518d5093" path="/var/lib/kubelet/pods/06c263f0-5102-4f0e-b30b-a797518d5093/volumes" Jul 21 18:40:25.877247 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:40:25.877223 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:40:25.881082 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:40:25.881063 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:41:05.613567 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.613476 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn"] Jul 21 18:41:05.614034 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.613928 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c263f0-5102-4f0e-b30b-a797518d5093" containerName="main" Jul 21 18:41:05.614034 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.613941 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c263f0-5102-4f0e-b30b-a797518d5093" containerName="main" Jul 21 18:41:05.614034 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.614004 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c263f0-5102-4f0e-b30b-a797518d5093" containerName="main" Jul 21 18:41:05.617300 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.617282 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.620979 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.620955 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"kube-root-ca.crt\"" Jul 21 18:41:05.621156 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.621138 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"default-dockercfg-fcm8s\"" Jul 21 18:41:05.621234 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.621164 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"auth-test-model-0be774-kserve-self-signed-certs\"" Jul 21 18:41:05.621234 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.621179 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"openshift-service-ca.crt\"" Jul 21 18:41:05.625953 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.625926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r42r\" (UniqueName: \"kubernetes.io/projected/b4eb5cb8-d700-4620-b4c3-87371215bf77-kube-api-access-2r42r\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.626077 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.626032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-model-cache\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.626077 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.626073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-home\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.626210 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.626184 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-dshm\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.626251 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.626223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eb5cb8-d700-4620-b4c3-87371215bf77-tls-certs\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.626320 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.626306 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-tmp-dir\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.627546 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.627516 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn"] Jul 21 18:41:05.726886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.726846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-model-cache\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.726886 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.726889 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-home\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727132 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.726926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-dshm\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727132 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.726942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eb5cb8-d700-4620-b4c3-87371215bf77-tls-certs\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727132 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.726979 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-tmp-dir\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727132 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.727010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r42r\" (UniqueName: \"kubernetes.io/projected/b4eb5cb8-d700-4620-b4c3-87371215bf77-kube-api-access-2r42r\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727336 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.727243 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-model-cache\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727336 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.727300 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-home\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.727429 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.727377 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-tmp-dir\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.729403 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.729376 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-dshm\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.729695 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.729678 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eb5cb8-d700-4620-b4c3-87371215bf77-tls-certs\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.736025 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.735997 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r42r\" (UniqueName: \"kubernetes.io/projected/b4eb5cb8-d700-4620-b4c3-87371215bf77-kube-api-access-2r42r\") pod \"auth-test-model-0be774-kserve-5c79c8b57f-m6znn\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:05.928714 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:05.928677 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:06.064094 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:06.064069 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn"] Jul 21 18:41:06.065997 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:41:06.065960 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4eb5cb8_d700_4620_b4c3_87371215bf77.slice/crio-63e5e9cd4cc843f8eda5b478a649f78157c1fcae1bbf7ed219aa5e7663c0e6ca WatchSource:0}: Error finding container 63e5e9cd4cc843f8eda5b478a649f78157c1fcae1bbf7ed219aa5e7663c0e6ca: Status 404 returned error can't find the container with id 63e5e9cd4cc843f8eda5b478a649f78157c1fcae1bbf7ed219aa5e7663c0e6ca Jul 21 18:41:06.259526 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:06.259408 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" event={"ID":"b4eb5cb8-d700-4620-b4c3-87371215bf77","Type":"ContainerStarted","Data":"e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6"} Jul 21 18:41:06.259526 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:06.259478 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" event={"ID":"b4eb5cb8-d700-4620-b4c3-87371215bf77","Type":"ContainerStarted","Data":"63e5e9cd4cc843f8eda5b478a649f78157c1fcae1bbf7ed219aa5e7663c0e6ca"} Jul 21 18:41:06.277832 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:06.277783 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" podStartSLOduration=1.277768462 podStartE2EDuration="1.277768462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:41:06.276183005 +0000 UTC m=+2441.348778703" watchObservedRunningTime="2026-07-21 18:41:06.277768462 +0000 UTC m=+2441.350364157" Jul 21 18:41:15.929684 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:15.929645 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:15.930147 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:15.929720 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:15.941971 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:15.941946 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:41:16.327219 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:41:16.327136 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:42:57.063881 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:42:57.063848 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn"] Jul 21 18:43:05.129897 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.129853 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh"] Jul 21 18:43:05.133622 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.133596 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.136092 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.136067 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"kube-root-ca.crt\"" Jul 21 18:43:05.137075 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.137056 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"openshift-service-ca.crt\"" Jul 21 18:43:05.137192 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.137109 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"default-dockercfg-j2vk6\"" Jul 21 18:43:05.137192 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.137110 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"sub-test-model-c1a1f4-kserve-self-signed-certs\"" Jul 21 18:43:05.142123 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.142097 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh"] Jul 21 18:43:05.213893 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.213854 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-model-cache\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.214058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.213927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-dshm\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.214058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.213965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-home\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.214058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.213988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.214058 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.214036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-tmp-dir\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.214189 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.214155 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zzgm\" (UniqueName: \"kubernetes.io/projected/0125d183-a832-460e-8691-3ccff197aa88-kube-api-access-8zzgm\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.314722 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.314672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zzgm\" (UniqueName: \"kubernetes.io/projected/0125d183-a832-460e-8691-3ccff197aa88-kube-api-access-8zzgm\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.314921 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.314740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-model-cache\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.314921 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.314779 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-dshm\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.314921 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.314815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-home\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.314921 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.314836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.314921 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.314857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-tmp-dir\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.315216 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.315194 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-model-cache\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.315279 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.315223 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-home\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.315324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.315279 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-tmp-dir\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.317329 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.317296 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-dshm\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.317635 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.317612 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.323493 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.323437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zzgm\" (UniqueName: \"kubernetes.io/projected/0125d183-a832-460e-8691-3ccff197aa88-kube-api-access-8zzgm\") pod \"sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.445464 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.445366 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:05.576412 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.576388 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh"] Jul 21 18:43:05.578804 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:43:05.578777 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0125d183_a832_460e_8691_3ccff197aa88.slice/crio-0fa05a910a580255b282ec9f10112826f339585fd80be814a958abaaf1ea5657 WatchSource:0}: Error finding container 0fa05a910a580255b282ec9f10112826f339585fd80be814a958abaaf1ea5657: Status 404 returned error can't find the container with id 0fa05a910a580255b282ec9f10112826f339585fd80be814a958abaaf1ea5657 Jul 21 18:43:05.850975 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.850887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" event={"ID":"0125d183-a832-460e-8691-3ccff197aa88","Type":"ContainerStarted","Data":"deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33"} Jul 21 18:43:05.850975 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.850929 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" event={"ID":"0125d183-a832-460e-8691-3ccff197aa88","Type":"ContainerStarted","Data":"0fa05a910a580255b282ec9f10112826f339585fd80be814a958abaaf1ea5657"} Jul 21 18:43:05.869010 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:05.868963 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" podStartSLOduration=0.868948184 podStartE2EDuration="868.948184ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:43:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:43:05.866911047 +0000 UTC m=+2560.939506743" watchObservedRunningTime="2026-07-21 18:43:05.868948184 +0000 UTC m=+2560.941543880" Jul 21 18:43:12.085694 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.085645 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" podUID="b4eb5cb8-d700-4620-b4c3-87371215bf77" containerName="main" containerID="cri-o://e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6" gracePeriod=45 Jul 21 18:43:12.336157 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.336092 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:43:12.485931 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.485887 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-model-cache\") pod \"b4eb5cb8-d700-4620-b4c3-87371215bf77\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " Jul 21 18:43:12.486121 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.485974 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-home\") pod \"b4eb5cb8-d700-4620-b4c3-87371215bf77\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " Jul 21 18:43:12.486121 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486002 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r42r\" (UniqueName: \"kubernetes.io/projected/b4eb5cb8-d700-4620-b4c3-87371215bf77-kube-api-access-2r42r\") pod \"b4eb5cb8-d700-4620-b4c3-87371215bf77\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " Jul 21 18:43:12.486121 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486035 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eb5cb8-d700-4620-b4c3-87371215bf77-tls-certs\") pod \"b4eb5cb8-d700-4620-b4c3-87371215bf77\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " Jul 21 18:43:12.486121 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486093 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-dshm\") pod \"b4eb5cb8-d700-4620-b4c3-87371215bf77\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " Jul 21 18:43:12.486320 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486164 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-tmp-dir\") pod \"b4eb5cb8-d700-4620-b4c3-87371215bf77\" (UID: \"b4eb5cb8-d700-4620-b4c3-87371215bf77\") " Jul 21 18:43:12.486320 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486204 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4eb5cb8-d700-4620-b4c3-87371215bf77" (UID: "b4eb5cb8-d700-4620-b4c3-87371215bf77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:43:12.486320 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486280 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-home" (OuterVolumeSpecName: "home") pod "b4eb5cb8-d700-4620-b4c3-87371215bf77" (UID: "b4eb5cb8-d700-4620-b4c3-87371215bf77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:43:12.486539 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486520 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4eb5cb8-d700-4620-b4c3-87371215bf77" (UID: "b4eb5cb8-d700-4620-b4c3-87371215bf77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:43:12.486619 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486604 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-tmp-dir\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:43:12.486673 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486627 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-model-cache\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:43:12.486673 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.486643 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-home\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:43:12.488508 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.488477 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4eb5cb8-d700-4620-b4c3-87371215bf77-kube-api-access-2r42r" (OuterVolumeSpecName: "kube-api-access-2r42r") pod "b4eb5cb8-d700-4620-b4c3-87371215bf77" (UID: "b4eb5cb8-d700-4620-b4c3-87371215bf77"). InnerVolumeSpecName "kube-api-access-2r42r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:43:12.488508 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.488490 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4eb5cb8-d700-4620-b4c3-87371215bf77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4eb5cb8-d700-4620-b4c3-87371215bf77" (UID: "b4eb5cb8-d700-4620-b4c3-87371215bf77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:43:12.488650 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.488524 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-dshm" (OuterVolumeSpecName: "dshm") pod "b4eb5cb8-d700-4620-b4c3-87371215bf77" (UID: "b4eb5cb8-d700-4620-b4c3-87371215bf77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:43:12.587976 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.587881 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eb5cb8-d700-4620-b4c3-87371215bf77-tls-certs\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:43:12.587976 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.587913 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eb5cb8-d700-4620-b4c3-87371215bf77-dshm\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:43:12.587976 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.587923 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r42r\" (UniqueName: \"kubernetes.io/projected/b4eb5cb8-d700-4620-b4c3-87371215bf77-kube-api-access-2r42r\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:43:12.886229 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.886196 2560 generic.go:356] "Generic (PLEG): container finished" podID="b4eb5cb8-d700-4620-b4c3-87371215bf77" containerID="e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6" exitCode=0 Jul 21 18:43:12.886373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.886241 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" Jul 21 18:43:12.886373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.886247 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" event={"ID":"b4eb5cb8-d700-4620-b4c3-87371215bf77","Type":"ContainerDied","Data":"e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6"} Jul 21 18:43:12.886373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.886287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn" event={"ID":"b4eb5cb8-d700-4620-b4c3-87371215bf77","Type":"ContainerDied","Data":"63e5e9cd4cc843f8eda5b478a649f78157c1fcae1bbf7ed219aa5e7663c0e6ca"} Jul 21 18:43:12.886373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.886309 2560 scope.go:117] "RemoveContainer" containerID="e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6" Jul 21 18:43:12.895606 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.895588 2560 scope.go:117] "RemoveContainer" containerID="e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6" Jul 21 18:43:12.895875 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:43:12.895854 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6\": container with ID starting with e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6 not found: ID does not exist" containerID="e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6" Jul 21 18:43:12.895937 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.895884 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6"} err="failed to get container status \"e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6\": rpc error: code = NotFound desc = could not find container \"e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6\": container with ID starting with e0e53c2fbd81857e219691ac4632f43251276645949e837713cd414287fa48d6 not found: ID does not exist" Jul 21 18:43:12.906946 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.906905 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn"] Jul 21 18:43:12.910617 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:12.910592 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/auth-test-model-0be774-kserve-5c79c8b57f-m6znn"] Jul 21 18:43:13.533356 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:13.533322 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4eb5cb8-d700-4620-b4c3-87371215bf77" path="/var/lib/kubelet/pods/b4eb5cb8-d700-4620-b4c3-87371215bf77/volumes" Jul 21 18:43:15.445629 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:15.445587 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:15.446057 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:15.445651 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:15.457773 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:15.457747 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:43:15.911878 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:43:15.911850 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:44:29.643000 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:29.642968 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh"] Jul 21 18:44:29.679044 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:29.679015 2560 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-self-signed-certs: secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:29.679193 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:29.679132 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs podName:0125d183-a832-460e-8691-3ccff197aa88 nodeName:}" failed. No retries permitted until 2026-07-21 18:44:30.179107673 +0000 UTC m=+2645.251703353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs") pod "sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" (UID: "0125d183-a832-460e-8691-3ccff197aa88") : secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:30.183394 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:30.183363 2560 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-self-signed-certs: secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:30.183565 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:30.183431 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs podName:0125d183-a832-460e-8691-3ccff197aa88 nodeName:}" failed. No retries permitted until 2026-07-21 18:44:31.183417502 +0000 UTC m=+2646.256013178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs") pod "sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" (UID: "0125d183-a832-460e-8691-3ccff197aa88") : secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:31.193788 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:31.193750 2560 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-self-signed-certs: secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:31.194188 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:31.193820 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs podName:0125d183-a832-460e-8691-3ccff197aa88 nodeName:}" failed. No retries permitted until 2026-07-21 18:44:33.193807392 +0000 UTC m=+2648.266403067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs") pod "sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" (UID: "0125d183-a832-460e-8691-3ccff197aa88") : secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:33.214016 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:33.213976 2560 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-self-signed-certs: secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:33.214413 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:33.214054 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs podName:0125d183-a832-460e-8691-3ccff197aa88 nodeName:}" failed. No retries permitted until 2026-07-21 18:44:37.214039555 +0000 UTC m=+2652.286635231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs") pod "sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" (UID: "0125d183-a832-460e-8691-3ccff197aa88") : secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:37.253666 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:37.253615 2560 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-self-signed-certs: secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:37.254214 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:37.253704 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs podName:0125d183-a832-460e-8691-3ccff197aa88 nodeName:}" failed. No retries permitted until 2026-07-21 18:44:45.253688835 +0000 UTC m=+2660.326284510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs") pod "sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" (UID: "0125d183-a832-460e-8691-3ccff197aa88") : secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:41.226048 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.226010 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf"] Jul 21 18:44:41.226464 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.226438 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4eb5cb8-d700-4620-b4c3-87371215bf77" containerName="main" Jul 21 18:44:41.226464 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.226466 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4eb5cb8-d700-4620-b4c3-87371215bf77" containerName="main" Jul 21 18:44:41.226547 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.226534 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4eb5cb8-d700-4620-b4c3-87371215bf77" containerName="main" Jul 21 18:44:41.229763 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.229743 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.232231 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.232213 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"rate-test-model-c1a1f4-kserve-self-signed-certs\"" Jul 21 18:44:41.238385 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.238352 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf"] Jul 21 18:44:41.394665 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.394624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-tmp-dir\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.394839 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.394685 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964e9451-d5f7-433c-86f5-696bf0f3724e-tls-certs\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.394839 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.394707 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbwt\" (UniqueName: \"kubernetes.io/projected/964e9451-d5f7-433c-86f5-696bf0f3724e-kube-api-access-ghbwt\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.394839 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.394753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-dshm\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.394839 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.394814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-model-cache\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.394994 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.394877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-home\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.495857 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.495778 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-tmp-dir\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.495857 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.495833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964e9451-d5f7-433c-86f5-696bf0f3724e-tls-certs\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.495857 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.495855 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbwt\" (UniqueName: \"kubernetes.io/projected/964e9451-d5f7-433c-86f5-696bf0f3724e-kube-api-access-ghbwt\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.496085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.495874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-dshm\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.496085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.495912 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-model-cache\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.496085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.495937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-home\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.496311 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.496286 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-tmp-dir\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.496382 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.496329 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-home\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.496431 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.496399 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-model-cache\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.498333 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.498316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-dshm\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.498644 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.498625 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964e9451-d5f7-433c-86f5-696bf0f3724e-tls-certs\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.505057 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.505037 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbwt\" (UniqueName: \"kubernetes.io/projected/964e9451-d5f7-433c-86f5-696bf0f3724e-kube-api-access-ghbwt\") pod \"rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.540035 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.540008 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:41.671588 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:41.671561 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf"] Jul 21 18:44:41.673869 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:44:41.673841 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod964e9451_d5f7_433c_86f5_696bf0f3724e.slice/crio-d4127d4e40114c7e557c73d9e61d3d231ebbbf006f36a59c53f26800b8565839 WatchSource:0}: Error finding container d4127d4e40114c7e557c73d9e61d3d231ebbbf006f36a59c53f26800b8565839: Status 404 returned error can't find the container with id d4127d4e40114c7e557c73d9e61d3d231ebbbf006f36a59c53f26800b8565839 Jul 21 18:44:42.313063 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:42.313032 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" event={"ID":"964e9451-d5f7-433c-86f5-696bf0f3724e","Type":"ContainerStarted","Data":"51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3"} Jul 21 18:44:42.313418 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:42.313071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" event={"ID":"964e9451-d5f7-433c-86f5-696bf0f3724e","Type":"ContainerStarted","Data":"d4127d4e40114c7e557c73d9e61d3d231ebbbf006f36a59c53f26800b8565839"} Jul 21 18:44:42.332116 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:42.332064 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" podStartSLOduration=1.332048315 podStartE2EDuration="1.332048315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:44:42.330125819 +0000 UTC m=+2657.402721517" watchObservedRunningTime="2026-07-21 18:44:42.332048315 +0000 UTC m=+2657.404644037" Jul 21 18:44:45.270851 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.270801 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" podUID="0125d183-a832-460e-8691-3ccff197aa88" containerName="main" containerID="cri-o://deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33" gracePeriod=45 Jul 21 18:44:45.331249 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:45.331216 2560 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-self-signed-certs: secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:45.331379 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:45.331288 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs podName:0125d183-a832-460e-8691-3ccff197aa88 nodeName:}" failed. No retries permitted until 2026-07-21 18:45:01.331274083 +0000 UTC m=+2676.403869758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs") pod "sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" (UID: "0125d183-a832-460e-8691-3ccff197aa88") : secret "sub-test-model-c1a1f4-kserve-self-signed-certs" not found Jul 21 18:44:45.522652 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.522592 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:44:45.532809 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.532785 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-dshm\") pod \"0125d183-a832-460e-8691-3ccff197aa88\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " Jul 21 18:44:45.532947 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.532815 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-tmp-dir\") pod \"0125d183-a832-460e-8691-3ccff197aa88\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " Jul 21 18:44:45.532947 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.532864 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs\") pod \"0125d183-a832-460e-8691-3ccff197aa88\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " Jul 21 18:44:45.532947 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.532904 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-home\") pod \"0125d183-a832-460e-8691-3ccff197aa88\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " Jul 21 18:44:45.532947 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.532935 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-model-cache\") pod \"0125d183-a832-460e-8691-3ccff197aa88\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " Jul 21 18:44:45.533150 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533044 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zzgm\" (UniqueName: \"kubernetes.io/projected/0125d183-a832-460e-8691-3ccff197aa88-kube-api-access-8zzgm\") pod \"0125d183-a832-460e-8691-3ccff197aa88\" (UID: \"0125d183-a832-460e-8691-3ccff197aa88\") " Jul 21 18:44:45.533150 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533125 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0125d183-a832-460e-8691-3ccff197aa88" (UID: "0125d183-a832-460e-8691-3ccff197aa88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:44:45.533255 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533153 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-home" (OuterVolumeSpecName: "home") pod "0125d183-a832-460e-8691-3ccff197aa88" (UID: "0125d183-a832-460e-8691-3ccff197aa88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:44:45.533309 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533260 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-model-cache" (OuterVolumeSpecName: "model-cache") pod "0125d183-a832-460e-8691-3ccff197aa88" (UID: "0125d183-a832-460e-8691-3ccff197aa88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:44:45.533357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533337 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-tmp-dir\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:44:45.533413 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533356 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-home\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:44:45.533413 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.533369 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-model-cache\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:44:45.535197 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.535174 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-dshm" (OuterVolumeSpecName: "dshm") pod "0125d183-a832-460e-8691-3ccff197aa88" (UID: "0125d183-a832-460e-8691-3ccff197aa88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:44:45.535197 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.535181 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0125d183-a832-460e-8691-3ccff197aa88-kube-api-access-8zzgm" (OuterVolumeSpecName: "kube-api-access-8zzgm") pod "0125d183-a832-460e-8691-3ccff197aa88" (UID: "0125d183-a832-460e-8691-3ccff197aa88"). InnerVolumeSpecName "kube-api-access-8zzgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:44:45.535197 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.535185 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0125d183-a832-460e-8691-3ccff197aa88" (UID: "0125d183-a832-460e-8691-3ccff197aa88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:44:45.634594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.634557 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0125d183-a832-460e-8691-3ccff197aa88-tls-certs\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:44:45.634594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.634585 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zzgm\" (UniqueName: \"kubernetes.io/projected/0125d183-a832-460e-8691-3ccff197aa88-kube-api-access-8zzgm\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:44:45.634594 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:45.634594 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0125d183-a832-460e-8691-3ccff197aa88-dshm\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:44:46.332372 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.332333 2560 generic.go:356] "Generic (PLEG): container finished" podID="0125d183-a832-460e-8691-3ccff197aa88" containerID="deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33" exitCode=0 Jul 21 18:44:46.332806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.332395 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" event={"ID":"0125d183-a832-460e-8691-3ccff197aa88","Type":"ContainerDied","Data":"deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33"} Jul 21 18:44:46.332806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.332399 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" Jul 21 18:44:46.332806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.332430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh" event={"ID":"0125d183-a832-460e-8691-3ccff197aa88","Type":"ContainerDied","Data":"0fa05a910a580255b282ec9f10112826f339585fd80be814a958abaaf1ea5657"} Jul 21 18:44:46.332806 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.332469 2560 scope.go:117] "RemoveContainer" containerID="deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33" Jul 21 18:44:46.341817 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.341795 2560 scope.go:117] "RemoveContainer" containerID="deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33" Jul 21 18:44:46.342060 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:44:46.342044 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33\": container with ID starting with deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33 not found: ID does not exist" containerID="deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33" Jul 21 18:44:46.342105 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.342066 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33"} err="failed to get container status \"deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33\": rpc error: code = NotFound desc = could not find container \"deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33\": container with ID starting with deec7d1b4f1356ad3ee29242a65a2e9df26e213efc0e045c08d1f979deaf9c33 not found: ID does not exist" Jul 21 18:44:46.354210 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.354188 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh"] Jul 21 18:44:46.358406 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:46.358383 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/sub-test-model-c1a1f4-kserve-d7588b6c9-sr6hh"] Jul 21 18:44:47.534615 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:47.534587 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0125d183-a832-460e-8691-3ccff197aa88" path="/var/lib/kubelet/pods/0125d183-a832-460e-8691-3ccff197aa88/volumes" Jul 21 18:44:51.540617 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:51.540586 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:51.541100 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:51.540631 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:51.553000 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:51.552975 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:44:52.373642 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:44:52.373612 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:45:25.911414 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:25.911299 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:45:25.916906 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:25.916347 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzqwp_5acea20a-96d0-4ebf-b6be-27366b1575ce/ovn-acl-logging/0.log" Jul 21 18:45:55.164409 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:55.164366 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf"] Jul 21 18:45:59.764373 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.764337 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7562-140036458843312"] Jul 21 18:45:59.764843 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.764765 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0125d183-a832-460e-8691-3ccff197aa88" containerName="main" Jul 21 18:45:59.764843 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.764778 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0125d183-a832-460e-8691-3ccff197aa88" containerName="main" Jul 21 18:45:59.764910 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.764849 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0125d183-a832-460e-8691-3ccff197aa88" containerName="main" Jul 21 18:45:59.768073 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.768057 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:45:59.770679 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.770650 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lfsn8\"" Jul 21 18:45:59.775020 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.774995 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458843312"] Jul 21 18:45:59.838740 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.838706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljd2\" (UniqueName: \"kubernetes.io/projected/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29-kube-api-access-rljd2\") pod \"test-curl-7562-140036458843312\" (UID: \"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29\") " pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:45:59.939258 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.939218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljd2\" (UniqueName: \"kubernetes.io/projected/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29-kube-api-access-rljd2\") pod \"test-curl-7562-140036458843312\" (UID: \"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29\") " pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:45:59.947771 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:45:59.947746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljd2\" (UniqueName: \"kubernetes.io/projected/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29-kube-api-access-rljd2\") pod \"test-curl-7562-140036458843312\" (UID: \"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29\") " pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:46:00.078487 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:00.078383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:46:00.214175 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:00.213737 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458843312"] Jul 21 18:46:00.220948 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:00.220926 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:46:00.687795 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:00.687759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458843312" event={"ID":"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29","Type":"ContainerStarted","Data":"67eeab92c72c54d73a71b184d531fdf4e87ae2ea4087d9747aeaca14b8d00cc7"} Jul 21 18:46:01.694852 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:01.694820 2560 generic.go:356] "Generic (PLEG): container finished" podID="3a35c769-c524-4c5c-bf41-5bc3eb1cdd29" containerID="4cb98d5816538742ff27977a9f190cf67ef00c4a5eddb302aad7cdd89130ce72" exitCode=0 Jul 21 18:46:01.695285 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:01.694888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458843312" event={"ID":"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29","Type":"ContainerDied","Data":"4cb98d5816538742ff27977a9f190cf67ef00c4a5eddb302aad7cdd89130ce72"} Jul 21 18:46:02.832048 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.832022 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:46:02.864623 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.864593 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rljd2\" (UniqueName: \"kubernetes.io/projected/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29-kube-api-access-rljd2\") pod \"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29\" (UID: \"3a35c769-c524-4c5c-bf41-5bc3eb1cdd29\") " Jul 21 18:46:02.866973 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.866943 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29-kube-api-access-rljd2" (OuterVolumeSpecName: "kube-api-access-rljd2") pod "3a35c769-c524-4c5c-bf41-5bc3eb1cdd29" (UID: "3a35c769-c524-4c5c-bf41-5bc3eb1cdd29"). InnerVolumeSpecName "kube-api-access-rljd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:02.867493 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.867475 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7562-140036458843312_3a35c769-c524-4c5c-bf41-5bc3eb1cdd29/test-curl-7562-140036458843312/0.log" Jul 21 18:46:02.894461 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.894414 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7562-140036458843312"] Jul 21 18:46:02.899944 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.899906 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7562-140036458843312"] Jul 21 18:46:02.966413 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:02.966326 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rljd2\" (UniqueName: \"kubernetes.io/projected/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29-kube-api-access-rljd2\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:03.533506 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:03.533467 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a35c769-c524-4c5c-bf41-5bc3eb1cdd29" path="/var/lib/kubelet/pods/3a35c769-c524-4c5c-bf41-5bc3eb1cdd29/volumes" Jul 21 18:46:03.708148 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:03.708116 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843312" Jul 21 18:46:03.708339 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:03.708168 2560 scope.go:117] "RemoveContainer" containerID="4cb98d5816538742ff27977a9f190cf67ef00c4a5eddb302aad7cdd89130ce72" Jul 21 18:46:08.826518 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.826479 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7562-140036458792976"] Jul 21 18:46:08.826934 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.826858 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a35c769-c524-4c5c-bf41-5bc3eb1cdd29" containerName="test-curl-7562-140036458843312" Jul 21 18:46:08.826934 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.826871 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a35c769-c524-4c5c-bf41-5bc3eb1cdd29" containerName="test-curl-7562-140036458843312" Jul 21 18:46:08.827001 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.826938 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a35c769-c524-4c5c-bf41-5bc3eb1cdd29" containerName="test-curl-7562-140036458843312" Jul 21 18:46:08.831299 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.831280 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:08.834061 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.834035 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lfsn8\"" Jul 21 18:46:08.835811 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.835786 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"] Jul 21 18:46:08.919561 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:08.919523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5s84\" (UniqueName: \"kubernetes.io/projected/ad070fac-6be1-4aad-b26c-4ed81991596d-kube-api-access-m5s84\") pod \"test-curl-7562-140036458792976\" (UID: \"ad070fac-6be1-4aad-b26c-4ed81991596d\") " pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:09.020996 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:09.020962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5s84\" (UniqueName: \"kubernetes.io/projected/ad070fac-6be1-4aad-b26c-4ed81991596d-kube-api-access-m5s84\") pod \"test-curl-7562-140036458792976\" (UID: \"ad070fac-6be1-4aad-b26c-4ed81991596d\") " pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:09.028885 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:09.028852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5s84\" (UniqueName: \"kubernetes.io/projected/ad070fac-6be1-4aad-b26c-4ed81991596d-kube-api-access-m5s84\") pod \"test-curl-7562-140036458792976\" (UID: \"ad070fac-6be1-4aad-b26c-4ed81991596d\") " pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:09.141601 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:09.141563 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:09.479726 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:09.479703 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"] Jul 21 18:46:09.482120 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:46:09.482094 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad070fac_6be1_4aad_b26c_4ed81991596d.slice/crio-ae3207b31cb5958398f69ee68cf17280c058520df2b26cb83402a07fc34c43f1 WatchSource:0}: Error finding container ae3207b31cb5958398f69ee68cf17280c058520df2b26cb83402a07fc34c43f1: Status 404 returned error can't find the container with id ae3207b31cb5958398f69ee68cf17280c058520df2b26cb83402a07fc34c43f1 Jul 21 18:46:09.738726 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:09.738645 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458792976" event={"ID":"ad070fac-6be1-4aad-b26c-4ed81991596d","Type":"ContainerStarted","Data":"ae3207b31cb5958398f69ee68cf17280c058520df2b26cb83402a07fc34c43f1"} Jul 21 18:46:10.185924 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.185885 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" podUID="964e9451-d5f7-433c-86f5-696bf0f3724e" containerName="main" containerID="cri-o://51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3" gracePeriod=45 Jul 21 18:46:10.446241 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.446216 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:46:10.535045 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535004 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-tmp-dir\") pod \"964e9451-d5f7-433c-86f5-696bf0f3724e\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " Jul 21 18:46:10.535045 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535048 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-dshm\") pod \"964e9451-d5f7-433c-86f5-696bf0f3724e\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " Jul 21 18:46:10.535297 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535070 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-home\") pod \"964e9451-d5f7-433c-86f5-696bf0f3724e\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " Jul 21 18:46:10.535297 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535122 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghbwt\" (UniqueName: \"kubernetes.io/projected/964e9451-d5f7-433c-86f5-696bf0f3724e-kube-api-access-ghbwt\") pod \"964e9451-d5f7-433c-86f5-696bf0f3724e\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " Jul 21 18:46:10.535297 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535188 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964e9451-d5f7-433c-86f5-696bf0f3724e-tls-certs\") pod \"964e9451-d5f7-433c-86f5-696bf0f3724e\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " Jul 21 18:46:10.535297 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535211 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-model-cache\") pod \"964e9451-d5f7-433c-86f5-696bf0f3724e\" (UID: \"964e9451-d5f7-433c-86f5-696bf0f3724e\") " Jul 21 18:46:10.535521 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535316 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "964e9451-d5f7-433c-86f5-696bf0f3724e" (UID: "964e9451-d5f7-433c-86f5-696bf0f3724e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:46:10.535521 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535370 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-home" (OuterVolumeSpecName: "home") pod "964e9451-d5f7-433c-86f5-696bf0f3724e" (UID: "964e9451-d5f7-433c-86f5-696bf0f3724e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:46:10.535521 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535477 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-model-cache" (OuterVolumeSpecName: "model-cache") pod "964e9451-d5f7-433c-86f5-696bf0f3724e" (UID: "964e9451-d5f7-433c-86f5-696bf0f3724e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:46:10.535639 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535579 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-tmp-dir\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:10.535639 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535598 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-home\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:10.535639 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.535610 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-model-cache\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:10.537416 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.537385 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-dshm" (OuterVolumeSpecName: "dshm") pod "964e9451-d5f7-433c-86f5-696bf0f3724e" (UID: "964e9451-d5f7-433c-86f5-696bf0f3724e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:46:10.537416 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.537393 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964e9451-d5f7-433c-86f5-696bf0f3724e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "964e9451-d5f7-433c-86f5-696bf0f3724e" (UID: "964e9451-d5f7-433c-86f5-696bf0f3724e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:46:10.537613 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.537435 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964e9451-d5f7-433c-86f5-696bf0f3724e-kube-api-access-ghbwt" (OuterVolumeSpecName: "kube-api-access-ghbwt") pod "964e9451-d5f7-433c-86f5-696bf0f3724e" (UID: "964e9451-d5f7-433c-86f5-696bf0f3724e"). InnerVolumeSpecName "kube-api-access-ghbwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:10.636085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.636052 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964e9451-d5f7-433c-86f5-696bf0f3724e-tls-certs\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:10.636085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.636082 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964e9451-d5f7-433c-86f5-696bf0f3724e-dshm\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:10.636085 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.636091 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghbwt\" (UniqueName: \"kubernetes.io/projected/964e9451-d5f7-433c-86f5-696bf0f3724e-kube-api-access-ghbwt\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:10.745068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.745024 2560 generic.go:356] "Generic (PLEG): container finished" podID="964e9451-d5f7-433c-86f5-696bf0f3724e" containerID="51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3" exitCode=0 Jul 21 18:46:10.745068 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.745072 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" Jul 21 18:46:10.745307 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.745118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" event={"ID":"964e9451-d5f7-433c-86f5-696bf0f3724e","Type":"ContainerDied","Data":"51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3"} Jul 21 18:46:10.745307 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.745172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf" event={"ID":"964e9451-d5f7-433c-86f5-696bf0f3724e","Type":"ContainerDied","Data":"d4127d4e40114c7e557c73d9e61d3d231ebbbf006f36a59c53f26800b8565839"} Jul 21 18:46:10.745307 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.745197 2560 scope.go:117] "RemoveContainer" containerID="51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3" Jul 21 18:46:10.746763 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.746739 2560 generic.go:356] "Generic (PLEG): container finished" podID="ad070fac-6be1-4aad-b26c-4ed81991596d" containerID="e383dbefeaa77df741fce8c73b78a3a15f7b0bdaa194213e2d7484bbe6f635e5" exitCode=0 Jul 21 18:46:10.746901 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.746844 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458792976" event={"ID":"ad070fac-6be1-4aad-b26c-4ed81991596d","Type":"ContainerDied","Data":"e383dbefeaa77df741fce8c73b78a3a15f7b0bdaa194213e2d7484bbe6f635e5"} Jul 21 18:46:10.754051 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.754034 2560 scope.go:117] "RemoveContainer" containerID="51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3" Jul 21 18:46:10.754291 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:46:10.754275 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3\": container with ID starting with 51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3 not found: ID does not exist" containerID="51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3" Jul 21 18:46:10.754341 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.754301 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3"} err="failed to get container status \"51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3\": rpc error: code = NotFound desc = could not find container \"51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3\": container with ID starting with 51d0c87b454eaa0a269cbac141e56058d782e29e9b3b3b63abe389fe60b27db3 not found: ID does not exist" Jul 21 18:46:10.774979 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.774943 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf"] Jul 21 18:46:10.778564 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:10.778535 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/rate-test-model-c1a1f4-kserve-5497cd8b9f-26tmf"] Jul 21 18:46:11.533873 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.533839 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="964e9451-d5f7-433c-86f5-696bf0f3724e" path="/var/lib/kubelet/pods/964e9451-d5f7-433c-86f5-696bf0f3724e/volumes" Jul 21 18:46:11.881134 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.881110 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:11.901284 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.901258 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7562-140036458792976_ad070fac-6be1-4aad-b26c-4ed81991596d/test-curl-7562-140036458792976/0.log" Jul 21 18:46:11.927250 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.927212 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"] Jul 21 18:46:11.929309 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.929285 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"] Jul 21 18:46:11.949792 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.949755 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5s84\" (UniqueName: \"kubernetes.io/projected/ad070fac-6be1-4aad-b26c-4ed81991596d-kube-api-access-m5s84\") pod \"ad070fac-6be1-4aad-b26c-4ed81991596d\" (UID: \"ad070fac-6be1-4aad-b26c-4ed81991596d\") " Jul 21 18:46:11.952009 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:11.951986 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad070fac-6be1-4aad-b26c-4ed81991596d-kube-api-access-m5s84" (OuterVolumeSpecName: "kube-api-access-m5s84") pod "ad070fac-6be1-4aad-b26c-4ed81991596d" (UID: "ad070fac-6be1-4aad-b26c-4ed81991596d"). InnerVolumeSpecName "kube-api-access-m5s84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:12.051502 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:12.051465 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5s84\" (UniqueName: \"kubernetes.io/projected/ad070fac-6be1-4aad-b26c-4ed81991596d-kube-api-access-m5s84\") on node \"ip-10-0-139-4.ec2.internal\" DevicePath \"\"" Jul 21 18:46:12.757946 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:12.757915 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae3207b31cb5958398f69ee68cf17280c058520df2b26cb83402a07fc34c43f1" Jul 21 18:46:12.757946 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:12.757943 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:12.767340 ip-10-0-139-4 kubenswrapper[2560]: E0721 18:46:12.767305 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7562-140036458792976\" is forbidden: User \"system:node:ip-10-0-139-4.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-4.ec2.internal' and this object" podUID="ad070fac-6be1-4aad-b26c-4ed81991596d" pod="opendatahub/test-curl-7562-140036458792976" Jul 21 18:46:13.533046 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:13.533003 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad070fac-6be1-4aad-b26c-4ed81991596d" path="/var/lib/kubelet/pods/ad070fac-6be1-4aad-b26c-4ed81991596d/volumes" Jul 21 18:46:59.871315 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:46:59.871218 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-744f66d67-27qz2_0a40bac6-ffd7-46d1-9a63-20b0468d63e7/authorino/0.log" Jul 21 18:47:06.155425 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:06.155397 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-pb58h_06d343d6-f9b6-472d-ab2a-b1c661ff50e1/copy-manifests/0.log" Jul 21 18:47:06.163047 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:06.163024 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-pb58h_06d343d6-f9b6-472d-ab2a-b1c661ff50e1/manager/0.log" Jul 21 18:47:08.440900 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:08.440870 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-744f66d67-27qz2_0a40bac6-ffd7-46d1-9a63-20b0468d63e7/authorino/0.log" Jul 21 18:47:08.782059 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:08.781978 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-w7mgb_539c1c1c-8444-407c-88f3-5a5c72c95a23/kuadrant-console-plugin/0.log" Jul 21 18:47:09.018231 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:09.018199 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q7vxk_5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc/manager/0.log" Jul 21 18:47:09.955273 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:09.955246 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-nwjwg_9fd43684-6823-40bd-9c6f-ebabf4bfb8f1/kube-auth-proxy/0.log" Jul 21 18:47:10.203076 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:10.203049 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gcnpz_0347d7e9-6450-45cb-b327-8dd62c0324ca/payload-pre-processing/0.log" Jul 21 18:47:10.322309 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:10.322228 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-s54wd_959c0599-2959-40ce-aaca-539d3681d648/payload-processing/0.log" Jul 21 18:47:11.016880 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:11.016849 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-q9nj4_99e68066-b2a9-441b-8dfe-05c2ea89cc02/main/0.log" Jul 21 18:47:11.364706 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:11.364622 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lwq4n_303d3d5c-78d8-496d-ad22-3e1704c56892/main/0.log" Jul 21 18:47:19.468693 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:19.468663 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kzvc2_ad0416fc-678d-4d6d-8715-24d76b208eb5/global-pull-secret-syncer/0.log" Jul 21 18:47:19.594830 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:19.594795 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q64wj_e07542b9-a38f-4338-8602-60373dc3d93a/konnectivity-agent/0.log" Jul 21 18:47:19.690907 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:19.690878 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-4.ec2.internal_143a5776505ae49b336b0210346bca91/haproxy/0.log" Jul 21 18:47:24.257937 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:24.257900 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-744f66d67-27qz2_0a40bac6-ffd7-46d1-9a63-20b0468d63e7/authorino/0.log" Jul 21 18:47:24.375546 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:24.375518 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-w7mgb_539c1c1c-8444-407c-88f3-5a5c72c95a23/kuadrant-console-plugin/0.log" Jul 21 18:47:24.571408 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:24.571328 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q7vxk_5cd833e8-77c1-4f1f-95de-2b86fe9ca4cc/manager/0.log" Jul 21 18:47:26.245105 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.245080 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/alertmanager/0.log" Jul 21 18:47:26.268900 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.268876 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/config-reloader/0.log" Jul 21 18:47:26.293417 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.293392 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/kube-rbac-proxy-web/0.log" Jul 21 18:47:26.316119 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.316094 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/kube-rbac-proxy/0.log" Jul 21 18:47:26.338924 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.338905 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/kube-rbac-proxy-metric/0.log" Jul 21 18:47:26.364197 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.364167 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/prom-label-proxy/0.log" Jul 21 18:47:26.387590 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.387568 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1448cc9-0af4-47aa-86f3-9e13c8a45f25/init-config-reloader/0.log" Jul 21 18:47:26.466567 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.466537 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-rzn86_ee88fd76-5152-4601-a70a-b041335879d1/kube-state-metrics/0.log" Jul 21 18:47:26.490324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.490297 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-rzn86_ee88fd76-5152-4601-a70a-b041335879d1/kube-rbac-proxy-main/0.log" Jul 21 18:47:26.512290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.512266 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-rzn86_ee88fd76-5152-4601-a70a-b041335879d1/kube-rbac-proxy-self/0.log" Jul 21 18:47:26.538352 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.538330 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c5b88859d-g2wgs_d2069994-290e-4e72-a12f-8e509e333ca7/metrics-server/0.log" Jul 21 18:47:26.564076 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.564057 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-qnmtc_ae64415c-2f98-4d23-854d-adc8fef068c9/monitoring-plugin/0.log" Jul 21 18:47:26.737334 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.737265 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sz72g_45beceea-a768-4f04-a69a-e54e0431fea6/node-exporter/0.log" Jul 21 18:47:26.758558 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.758535 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sz72g_45beceea-a768-4f04-a69a-e54e0431fea6/kube-rbac-proxy/0.log" Jul 21 18:47:26.787440 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.787418 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sz72g_45beceea-a768-4f04-a69a-e54e0431fea6/init-textfile/0.log" Jul 21 18:47:26.817336 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.817313 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zznq7_80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181/kube-rbac-proxy-main/0.log" Jul 21 18:47:26.841881 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.841858 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zznq7_80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181/kube-rbac-proxy-self/0.log" Jul 21 18:47:26.908629 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.908605 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zznq7_80d63e9c-bbcb-4dce-b8e5-eb8b5adf4181/openshift-state-metrics/0.log" Jul 21 18:47:26.963173 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.963149 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/prometheus/0.log" Jul 21 18:47:26.995222 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:26.995205 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/config-reloader/0.log" Jul 21 18:47:27.023263 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.023242 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/thanos-sidecar/0.log" Jul 21 18:47:27.063842 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.063816 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/kube-rbac-proxy-web/0.log" Jul 21 18:47:27.088872 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.088845 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/kube-rbac-proxy/0.log" Jul 21 18:47:27.118850 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.118831 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/kube-rbac-proxy-thanos/0.log" Jul 21 18:47:27.145600 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.145578 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5f818b7c-d9b2-446d-a6d4-d38c6cf167b5/init-config-reloader/0.log" Jul 21 18:47:27.177729 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.177704 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-qb8vw_5eca6724-e762-40b4-ac32-e6805163e231/prometheus-operator/0.log" Jul 21 18:47:27.201935 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.201915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-qb8vw_5eca6724-e762-40b4-ac32-e6805163e231/kube-rbac-proxy/0.log" Jul 21 18:47:27.232432 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.232410 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-59jkb_72ebc1e3-2eb4-4b9a-a099-4757a8f71ef1/prometheus-operator-admission-webhook/0.log" Jul 21 18:47:27.375290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.375263 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b4658575-flbgw_9536aa4a-a08d-47dc-a51e-b3128755c993/thanos-query/0.log" Jul 21 18:47:27.405026 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.404998 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b4658575-flbgw_9536aa4a-a08d-47dc-a51e-b3128755c993/kube-rbac-proxy-web/0.log" Jul 21 18:47:27.431535 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.431511 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b4658575-flbgw_9536aa4a-a08d-47dc-a51e-b3128755c993/kube-rbac-proxy/0.log" Jul 21 18:47:27.457767 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.457743 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b4658575-flbgw_9536aa4a-a08d-47dc-a51e-b3128755c993/prom-label-proxy/0.log" Jul 21 18:47:27.481044 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.481022 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b4658575-flbgw_9536aa4a-a08d-47dc-a51e-b3128755c993/kube-rbac-proxy-rules/0.log" Jul 21 18:47:27.510398 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:27.510371 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b4658575-flbgw_9536aa4a-a08d-47dc-a51e-b3128755c993/kube-rbac-proxy-metrics/0.log" Jul 21 18:47:30.100698 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.100668 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch"] Jul 21 18:47:30.101239 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.101219 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad070fac-6be1-4aad-b26c-4ed81991596d" containerName="test-curl-7562-140036458792976" Jul 21 18:47:30.101324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.101241 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad070fac-6be1-4aad-b26c-4ed81991596d" containerName="test-curl-7562-140036458792976" Jul 21 18:47:30.101324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.101272 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="964e9451-d5f7-433c-86f5-696bf0f3724e" containerName="main" Jul 21 18:47:30.101324 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.101281 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="964e9451-d5f7-433c-86f5-696bf0f3724e" containerName="main" Jul 21 18:47:30.101493 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.101387 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="964e9451-d5f7-433c-86f5-696bf0f3724e" containerName="main" Jul 21 18:47:30.101493 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.101399 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad070fac-6be1-4aad-b26c-4ed81991596d" containerName="test-curl-7562-140036458792976" Jul 21 18:47:30.104785 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.104760 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.107630 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.107604 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg7gx\"/\"kube-root-ca.crt\"" Jul 21 18:47:30.107763 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.107720 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg7gx\"/\"openshift-service-ca.crt\"" Jul 21 18:47:30.116410 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.116372 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gg7gx\"/\"default-dockercfg-pzzcs\"" Jul 21 18:47:30.131297 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.131271 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch"] Jul 21 18:47:30.274637 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.274600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv27h\" (UniqueName: \"kubernetes.io/projected/ebad36e0-b104-4587-87c2-e9613da06eb0-kube-api-access-vv27h\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.274819 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.274683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-lib-modules\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.274819 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.274735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-podres\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.274819 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.274799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-proc\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.274947 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.274851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-sys\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375668 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375638 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv27h\" (UniqueName: \"kubernetes.io/projected/ebad36e0-b104-4587-87c2-e9613da06eb0-kube-api-access-vv27h\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375820 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-lib-modules\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375820 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-podres\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375820 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-proc\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375820 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-sys\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375978 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-proc\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375978 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375898 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-lib-modules\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375978 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375881 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-podres\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.375978 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.375878 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebad36e0-b104-4587-87c2-e9613da06eb0-sys\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.385784 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.385758 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv27h\" (UniqueName: \"kubernetes.io/projected/ebad36e0-b104-4587-87c2-e9613da06eb0-kube-api-access-vv27h\") pod \"perf-node-gather-daemonset-grtch\" (UID: \"ebad36e0-b104-4587-87c2-e9613da06eb0\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.417747 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.417722 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:30.548752 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:30.548721 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch"] Jul 21 18:47:30.552590 ip-10-0-139-4 kubenswrapper[2560]: W0721 18:47:30.552568 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podebad36e0_b104_4587_87c2_e9613da06eb0.slice/crio-90f07278187e39783d5b86e1d855d5d24e33a65db62cd90f734dfaf79aa4c6bb WatchSource:0}: Error finding container 90f07278187e39783d5b86e1d855d5d24e33a65db62cd90f734dfaf79aa4c6bb: Status 404 returned error can't find the container with id 90f07278187e39783d5b86e1d855d5d24e33a65db62cd90f734dfaf79aa4c6bb Jul 21 18:47:31.141131 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:31.141098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" event={"ID":"ebad36e0-b104-4587-87c2-e9613da06eb0","Type":"ContainerStarted","Data":"30d173a979b6e2e886eb67d8d42f83dc1c66cb214738de15e04e9bfc19fbd492"} Jul 21 18:47:31.141659 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:31.141147 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" event={"ID":"ebad36e0-b104-4587-87c2-e9613da06eb0","Type":"ContainerStarted","Data":"90f07278187e39783d5b86e1d855d5d24e33a65db62cd90f734dfaf79aa4c6bb"} Jul 21 18:47:31.141659 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:31.141224 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:31.158835 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:31.158789 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" podStartSLOduration=1.158774469 podStartE2EDuration="1.158774469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:47:31.157123863 +0000 UTC m=+2826.229719559" watchObservedRunningTime="2026-07-21 18:47:31.158774469 +0000 UTC m=+2826.231370165" Jul 21 18:47:31.973000 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:31.972975 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z677t_9e537933-d2d1-4430-bea8-23b23547766a/dns/0.log" Jul 21 18:47:31.994703 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:31.994675 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z677t_9e537933-d2d1-4430-bea8-23b23547766a/kube-rbac-proxy/0.log" Jul 21 18:47:32.045917 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:32.045878 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rvrzw_e85ae40c-e1d2-45cd-87c5-4c63e711ef78/dns-node-resolver/0.log" Jul 21 18:47:32.707141 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:32.707113 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bf96n_15cb2308-3ce4-450f-a0c9-4c391412109b/node-ca/0.log" Jul 21 18:47:33.882336 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:33.882307 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-nwjwg_9fd43684-6823-40bd-9c6f-ebabf4bfb8f1/kube-auth-proxy/0.log" Jul 21 18:47:34.001374 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:34.001346 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gcnpz_0347d7e9-6450-45cb-b327-8dd62c0324ca/payload-pre-processing/0.log" Jul 21 18:47:34.032217 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:34.032191 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-s54wd_959c0599-2959-40ce-aaca-539d3681d648/payload-processing/0.log" Jul 21 18:47:34.725307 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:34.725278 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2dh7q_623b73a6-41c3-4b45-86ba-f075aeaa1dee/serve-healthcheck-canary/0.log" Jul 21 18:47:35.389357 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:35.389327 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsb48_6bb6b80d-0555-4df3-a29a-bbfe17b96ab0/kube-rbac-proxy/0.log" Jul 21 18:47:35.411536 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:35.411508 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsb48_6bb6b80d-0555-4df3-a29a-bbfe17b96ab0/exporter/0.log" Jul 21 18:47:35.433715 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:35.433688 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsb48_6bb6b80d-0555-4df3-a29a-bbfe17b96ab0/extractor/0.log" Jul 21 18:47:37.155733 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:37.155704 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-grtch" Jul 21 18:47:38.345530 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:38.345505 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-pb58h_06d343d6-f9b6-472d-ab2a-b1c661ff50e1/manager/0.log" Jul 21 18:47:38.366258 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:38.366234 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-pb58h_06d343d6-f9b6-472d-ab2a-b1c661ff50e1/copy-manifests/0.log" Jul 21 18:47:40.056822 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:40.056792 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-nwjwg_9fd43684-6823-40bd-9c6f-ebabf4bfb8f1/kube-auth-proxy/0.log" Jul 21 18:47:40.074459 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:40.074427 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-754c46d99f-bvfms_e6631fc4-3caf-4b06-85c2-3df49f258357/manager/0.log" Jul 21 18:47:40.107490 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:40.107440 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fz6tp_e1bd2d95-6a07-45e8-a6cd-b9938516b24c/openshift-lws-operator/0.log" Jul 21 18:47:40.160562 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:40.160532 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gcnpz_0347d7e9-6450-45cb-b327-8dd62c0324ca/payload-pre-processing/0.log" Jul 21 18:47:40.185830 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:40.185805 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-s54wd_959c0599-2959-40ce-aaca-539d3681d648/payload-processing/0.log" Jul 21 18:47:41.611500 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:41.611473 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-nwjwg_9fd43684-6823-40bd-9c6f-ebabf4bfb8f1/kube-auth-proxy/0.log" Jul 21 18:47:41.713976 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:41.713942 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-gcnpz_0347d7e9-6450-45cb-b327-8dd62c0324ca/payload-pre-processing/0.log" Jul 21 18:47:41.732233 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:41.732206 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-s54wd_959c0599-2959-40ce-aaca-539d3681d648/payload-processing/0.log" Jul 21 18:47:46.402073 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.402044 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/kube-multus-additional-cni-plugins/0.log" Jul 21 18:47:46.422773 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.422747 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/egress-router-binary-copy/0.log" Jul 21 18:47:46.442440 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.442418 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/cni-plugins/0.log" Jul 21 18:47:46.465501 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.465478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/bond-cni-plugin/0.log" Jul 21 18:47:46.487092 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.487072 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/routeoverride-cni/0.log" Jul 21 18:47:46.507587 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.507568 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/whereabouts-cni-bincopy/0.log" Jul 21 18:47:46.527040 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.527021 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dcrrq_0fdef49f-195a-4d8f-9e49-6fbb46a08a84/whereabouts-cni/0.log" Jul 21 18:47:46.933252 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.933221 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f85qv_964214a8-2f78-4f76-bd48-dc273435e190/kube-multus/0.log" Jul 21 18:47:46.995290 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:46.995261 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ltwt9_225d56ab-f30e-44dc-bc64-42071d0e90b8/network-metrics-daemon/0.log" Jul 21 18:47:47.016434 ip-10-0-139-4 kubenswrapper[2560]: I0721 18:47:47.016404 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ltwt9_225d56ab-f30e-44dc-bc64-42071d0e90b8/kube-rbac-proxy/0.log"