Jul 08 21:30:51.204653 ip-10-0-130-8 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 21:30:51.204661 ip-10-0-130-8 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 21:30:51.204671 ip-10-0-130-8 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 21:30:51.205000 ip-10-0-130-8 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 21:31:01.307684 ip-10-0-130-8 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 21:31:01.307704 ip-10-0-130-8 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a09795197e2d47c985de7ce74534c459 -- Jul 08 21:32:56.674443 ip-10-0-130-8 systemd[1]: Starting Kubernetes Kubelet... Jul 08 21:32:57.012928 ip-10-0-130-8 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:32:57.012928 ip-10-0-130-8 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 21:32:57.012928 ip-10-0-130-8 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:32:57.012928 ip-10-0-130-8 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 21:32:57.012928 ip-10-0-130-8 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:32:57.014229 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.014100 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 21:32:57.018382 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018364 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.018382 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018382 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018386 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018389 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018393 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018395 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018398 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018403 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018405 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018408 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018411 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018413 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018416 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018418 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018421 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018424 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018427 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018430 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018435 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018439 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018443 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.018451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018446 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018449 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018453 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018456 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018460 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018463 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018467 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018471 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018474 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018477 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018479 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018482 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018485 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018488 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018491 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018493 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018496 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018498 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018501 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.018921 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018504 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018507 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018510 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018513 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018515 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018518 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018520 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018522 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018525 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018527 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018530 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018533 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018535 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018538 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018540 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018544 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018547 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018549 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018552 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.019395 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018554 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018557 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018559 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018561 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018564 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018567 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018569 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018572 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018574 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018576 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018581 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018583 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018587 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018589 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018592 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018594 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018597 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018599 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018602 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018604 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.019872 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018607 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018610 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018612 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018614 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018617 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018619 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018622 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018624 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018628 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018631 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018633 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018636 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018638 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018641 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018643 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018646 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018648 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018651 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018653 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.018656 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.020371 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019375 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019382 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019384 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019387 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019390 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019393 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019396 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019399 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019404 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019408 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019412 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019416 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019419 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019422 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019425 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019428 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019431 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019433 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019435 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.020879 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019438 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019441 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019444 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019447 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019449 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019452 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019455 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019457 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019460 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019462 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019464 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019467 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019469 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019472 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019474 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019476 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019479 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019481 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019484 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019487 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.021370 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019489 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019492 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019495 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019497 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019500 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019502 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019504 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019507 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019510 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019512 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019514 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019517 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019519 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019522 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019525 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019527 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019530 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019532 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019534 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019537 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.021855 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019539 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019541 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019544 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019546 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019548 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019551 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019553 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019556 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019558 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019561 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019563 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019567 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019571 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019574 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019577 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019579 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019583 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019586 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019589 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.022366 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019591 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019594 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019597 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019599 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019602 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019605 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019608 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019611 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019613 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019616 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019619 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019621 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019624 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019627 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019629 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019632 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019635 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019638 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019640 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019643 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.022832 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.019645 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020022 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020027 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020030 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020032 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020035 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020038 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020041 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020044 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020049 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020052 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020054 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020057 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020059 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020062 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020064 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020067 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020070 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020072 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020074 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.023321 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020077 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020079 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020082 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020084 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020087 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020089 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020092 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020094 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020097 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020099 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020102 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020104 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020106 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020109 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020111 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020113 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020116 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020118 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020136 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020140 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.023797 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020143 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020147 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020150 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020152 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020155 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020157 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020160 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020162 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020165 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020167 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020170 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020172 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020176 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020179 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020182 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020185 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020187 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020189 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020192 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.024319 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020194 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020197 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020199 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020201 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020204 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020206 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020209 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020211 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020213 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020216 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020218 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020221 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020225 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020230 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020235 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020238 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020241 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020243 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020246 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.024768 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020248 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020251 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020253 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020255 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020258 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020260 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020263 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020265 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020268 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020270 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020273 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020275 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020277 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020280 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020282 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020285 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020287 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020289 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020292 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020294 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.025260 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020296 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.020300 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020376 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020383 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020389 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020395 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020400 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020404 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020408 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020412 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020415 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020418 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020421 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020425 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020427 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020430 2563 flags.go:64] FLAG: --cgroup-root="" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020433 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020436 2563 flags.go:64] FLAG: --client-ca-file="" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020439 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020442 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020446 2563 flags.go:64] FLAG: --cluster-domain="" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020449 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020452 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 21:32:57.025773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020455 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020459 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020462 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020465 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020468 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020471 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020473 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020476 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020479 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020482 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020485 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020489 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020492 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020495 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020500 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020503 2563 flags.go:64] FLAG: --enable-server="true" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020507 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020512 2563 flags.go:64] FLAG: --event-burst="100" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020515 2563 flags.go:64] FLAG: --event-qps="50" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020518 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020521 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020524 2563 flags.go:64] FLAG: --eviction-hard="" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020528 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020530 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020533 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 21:32:57.026340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020536 2563 flags.go:64] FLAG: --eviction-soft="" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020539 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020541 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020544 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020547 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020550 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020552 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020555 2563 flags.go:64] FLAG: --feature-gates="" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020558 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020561 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020564 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020567 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020570 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020573 2563 flags.go:64] FLAG: --help="false" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020576 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020579 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020581 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020600 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020605 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020609 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020612 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020616 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020619 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 21:32:57.027003 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020622 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020625 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020628 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020632 2563 flags.go:64] FLAG: --kube-reserved="" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020634 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020637 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020640 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020643 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020645 2563 flags.go:64] FLAG: --lock-file="" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020648 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020651 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020653 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020659 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020661 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020664 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020667 2563 flags.go:64] FLAG: --logging-format="text" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020670 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020673 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020675 2563 flags.go:64] FLAG: --manifest-url="" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020678 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020682 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020685 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020689 2563 flags.go:64] FLAG: --max-pods="110" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020692 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020695 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 21:32:57.027641 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020697 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020700 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020703 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020706 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020709 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020717 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020720 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020723 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020726 2563 flags.go:64] FLAG: --pod-cidr="" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020729 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020738 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020741 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020744 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020747 2563 flags.go:64] FLAG: --port="10250" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020750 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020753 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035da8d467f72a05c" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020756 2563 flags.go:64] FLAG: --qos-reserved="" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020758 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020761 2563 flags.go:64] FLAG: --register-node="true" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020764 2563 flags.go:64] FLAG: --register-with-taints="" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020767 2563 flags.go:64] FLAG: --registry-burst="10" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020770 2563 flags.go:64] FLAG: --registry-qps="5" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020772 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020775 2563 flags.go:64] FLAG: --reserved-memory="" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020779 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 21:32:57.028271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020781 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020784 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020787 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020790 2563 flags.go:64] FLAG: --runonce="false" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020792 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020795 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020798 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020801 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020804 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020806 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020809 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020812 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020814 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020819 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020823 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020825 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020837 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020846 2563 flags.go:64] FLAG: --system-cgroups="" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020849 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020855 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020858 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020860 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020867 2563 flags.go:64] FLAG: --tls-min-version="" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020870 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020872 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 21:32:57.028887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020875 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020878 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020881 2563 flags.go:64] FLAG: --v="2" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020885 2563 flags.go:64] FLAG: --version="false" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020889 2563 flags.go:64] FLAG: --vmodule="" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020893 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.020896 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021021 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021025 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021028 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021031 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021034 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021037 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021040 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021043 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021045 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021048 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021051 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021053 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021056 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021061 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.029522 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021066 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021069 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021072 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021074 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021078 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021084 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021086 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021089 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021092 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021095 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021097 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021100 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021102 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021104 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021107 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021109 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021112 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021114 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021116 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021119 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.030047 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021134 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021137 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021139 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021142 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021144 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021147 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021149 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021152 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021154 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021157 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021159 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021164 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021169 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021171 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021174 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021177 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021180 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021184 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021186 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.030789 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021189 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021192 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021194 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021196 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021199 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021202 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021204 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021206 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021209 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021211 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021213 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021216 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021218 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021221 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021223 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021226 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021229 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021232 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021235 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021238 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.031569 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021240 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021243 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021245 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021248 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021251 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021256 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021258 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021261 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021263 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021266 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021268 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021272 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021274 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021277 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021279 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021282 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021284 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021287 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021289 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021292 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.032098 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021294 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021297 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021299 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021301 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021304 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.021306 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.021917 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.029019 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.029041 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.029717 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.029725 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029789 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029796 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029800 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029803 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.032602 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029806 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029809 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029812 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029815 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029818 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029821 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029824 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029827 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029830 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029833 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029837 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029841 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029843 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029846 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029848 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029851 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029854 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029857 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029860 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.032985 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029864 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029870 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029874 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029877 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029880 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029883 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029887 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029890 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029893 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029895 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029898 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029901 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029904 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029907 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029910 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029912 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029915 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029917 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029920 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.033471 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029922 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029924 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029927 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029929 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029932 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029935 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029938 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029942 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029946 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029950 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029954 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029956 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029958 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029961 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029963 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029966 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029968 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029972 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029975 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029978 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.033926 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029981 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029983 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029985 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029988 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029990 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029993 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029995 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.029998 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030000 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030002 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030005 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030008 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030010 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030012 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030016 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030020 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030024 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030029 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030033 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.034417 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030037 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030039 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030042 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030044 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030047 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030049 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030052 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030055 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030057 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030060 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030063 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030066 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030069 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030072 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030075 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030078 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030081 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.034869 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.030083 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.030088 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031305 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031327 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031333 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031339 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031344 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031348 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031353 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031357 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031361 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031371 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031374 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031379 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031383 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.035396 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031387 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031392 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031396 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031400 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031404 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031408 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031412 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031416 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031443 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031454 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031459 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031464 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031469 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031474 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031479 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031486 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031493 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031500 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031505 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031510 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.035767 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031542 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031559 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031566 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031686 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031693 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031699 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031703 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031706 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031709 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031712 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031715 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031717 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031720 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031723 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031726 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031728 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031731 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031734 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031746 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.036274 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031749 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031754 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031759 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031762 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031770 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031773 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031776 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031779 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031781 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031784 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031786 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031789 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031793 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031797 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031800 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031803 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031806 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031809 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031812 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.036729 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031815 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031817 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031820 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031824 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031827 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031830 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031833 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031836 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031838 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031841 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031844 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031847 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031850 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031852 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031855 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031858 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031861 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031863 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031866 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.037200 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031869 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031871 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031874 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031877 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031880 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031882 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031885 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031888 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:57.031890 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.031896 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.032045 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.034238 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.035496 2563 server.go:1019] "Starting client certificate rotation" Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.035594 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:32:57.037661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.035630 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:32:57.037999 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.037756 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:32:57.042626 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.042611 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:32:57.052259 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.052242 2563 log.go:25] "Validated CRI v1 runtime API" Jul 08 21:32:57.055894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.055875 2563 log.go:25] "Validated CRI v1 image API" Jul 08 21:32:57.056602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.056588 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 21:32:57.058713 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.058694 2563 fs.go:135] Filesystem UUIDs: map[0ac508d0-8c47-4638-8722-e88cfdda84fe:/dev/nvme0n1p3 14397b77-6531-4d8c-9d13-8a953779e2cc:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 08 21:32:57.058770 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.058713 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 21:32:57.061160 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.061145 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:32:57.064558 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.064449 2563 manager.go:216] Machine: {Timestamp:2026-07-08 21:32:57.063248015 +0000 UTC m=+0.285258207 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3083315 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cbc1e61947f4b1b3a7e93c9f63db7 SystemUUID:ec2cbc1e-6194-7f4b-1b3a-7e93c9f63db7 BootID:a0979519-7e2d-47c9-85de-7ce74534c459 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7e:e5:be:97:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7e:e5:be:97:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:8c:09:fb:02:6c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 21:32:57.064558 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.064554 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 21:32:57.064680 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.064669 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 21:32:57.067391 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.067366 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 21:32:57.067537 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.067394 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-8.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 21:32:57.068509 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.068499 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 21:32:57.068544 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.068511 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 21:32:57.068544 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.068518 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:32:57.069064 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.069053 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:32:57.070415 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.070404 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 21:32:57.074174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074161 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:32:57.074295 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074285 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 21:32:57.074325 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074317 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 08 21:32:57.074354 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074327 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 21:32:57.074354 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074338 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 21:32:57.074354 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074348 2563 kubelet.go:398] "Adding apiserver pod source" Jul 08 21:32:57.074443 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074357 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 21:32:57.074747 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074724 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:32:57.074747 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.074745 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:32:57.077254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.077224 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 21:32:57.079305 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.079288 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 21:32:57.079378 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.079320 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 21:32:57.080461 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080450 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080467 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080474 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080479 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080488 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080493 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080502 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 21:32:57.080504 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080507 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 21:32:57.080673 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080514 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 21:32:57.080673 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080519 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 21:32:57.080673 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080528 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 21:32:57.080673 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.080533 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 21:32:57.081162 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.081153 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 21:32:57.081162 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.081162 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 21:32:57.084505 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.084491 2563 server.go:1268] "Started kubelet" Jul 08 21:32:57.084624 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.084578 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 21:32:57.084740 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.084697 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 21:32:57.084801 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.084767 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 21:32:57.085815 ip-10-0-130-8 systemd[1]: Started Kubernetes Kubelet. Jul 08 21:32:57.086938 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.086919 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 08 21:32:57.087025 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.087005 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 21:32:57.089107 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.089087 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-8.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 21:32:57.089224 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.089210 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 21:32:57.089392 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.089373 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 21:32:57.089437 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.089375 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 21:32:57.089964 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.089111 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-8.ec2.internal.18c06dfaf68dd84a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-8.ec2.internal,UID:ip-10-0-130-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-8.ec2.internal,},FirstTimestamp:2026-07-08 21:32:57.084459082 +0000 UTC m=+0.306469273,LastTimestamp:2026-07-08 21:32:57.084459082 +0000 UTC m=+0.306469273,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-8.ec2.internal,}" Jul 08 21:32:57.093436 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.093421 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 21:32:57.093436 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.093434 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 21:32:57.093544 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.093526 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.094007 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.093980 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 21:32:57.094007 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.094000 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 21:32:57.094173 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.094162 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 21:32:57.094346 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.094330 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 08 21:32:57.094346 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.094346 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 08 21:32:57.094875 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.094849 2563 factory.go:55] Registering systemd factory Jul 08 21:32:57.094875 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.094867 2563 factory.go:223] Registration of the systemd container factory successfully Jul 08 21:32:57.095078 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.095067 2563 factory.go:153] Registering CRI-O factory Jul 08 21:32:57.095151 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.095080 2563 factory.go:223] Registration of the crio container factory successfully Jul 08 21:32:57.095215 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.095151 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 21:32:57.095215 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.095176 2563 factory.go:103] Registering Raw factory Jul 08 21:32:57.095215 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.095190 2563 manager.go:1196] Started watching for new ooms in manager Jul 08 21:32:57.095659 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.095607 2563 manager.go:349] Starting recovery of all containers Jul 08 21:32:57.104374 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.104347 2563 manager.go:354] Recovery completed Jul 08 21:32:57.105944 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.105777 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 21:32:57.105944 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.105933 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 21:32:57.109439 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.109424 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.111374 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111350 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.111448 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111390 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.111448 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111403 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.111944 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111925 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 21:32:57.111944 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111939 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 21:32:57.112068 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111954 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:32:57.112068 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.111966 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.112675 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.112660 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.112749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.112692 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.112749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.112705 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.113938 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.113865 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-8.ec2.internal.18c06dfaf82885e6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-8.ec2.internal,UID:ip-10-0-130-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-8.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-8.ec2.internal,},FirstTimestamp:2026-07-08 21:32:57.111373286 +0000 UTC m=+0.333383477,LastTimestamp:2026-07-08 21:32:57.111373286 +0000 UTC m=+0.333383477,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-8.ec2.internal,}" Jul 08 21:32:57.114264 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.114250 2563 policy_none.go:49] "None policy: Start" Jul 08 21:32:57.114326 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.114269 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 21:32:57.114326 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.114281 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 21:32:57.116252 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.116215 2563 policy_none.go:47] "Start" Jul 08 21:32:57.116417 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.116402 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2htll" Jul 08 21:32:57.121629 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.121610 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2htll" Jul 08 21:32:57.122410 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.122323 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-8.ec2.internal.18c06dfaf828e407 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-8.ec2.internal,UID:ip-10-0-130-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-8.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-8.ec2.internal,},FirstTimestamp:2026-07-08 21:32:57.111397383 +0000 UTC m=+0.333407574,LastTimestamp:2026-07-08 21:32:57.111397383 +0000 UTC m=+0.333407574,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-8.ec2.internal,}" Jul 08 21:32:57.148756 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.148735 2563 manager.go:337] "Starting Device Plugin manager" Jul 08 21:32:57.148898 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.148783 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 21:32:57.148898 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.148796 2563 server.go:87] "Starting device plugin registration server" Jul 08 21:32:57.149100 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.149085 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 21:32:57.149190 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.149101 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 21:32:57.149236 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.149212 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 21:32:57.149320 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.149308 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 21:32:57.149372 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.149321 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 21:32:57.149751 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.149728 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 21:32:57.149844 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.149786 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.242720 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.242685 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 21:32:57.245019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.244112 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 21:32:57.245019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.244172 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 21:32:57.245019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.244201 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 21:32:57.245019 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.244247 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 21:32:57.246343 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.246325 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:57.249281 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.249265 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.250045 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.250028 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.250097 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.250062 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.250097 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.250072 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.250097 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.250094 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.258245 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.258226 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.258323 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.258251 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-8.ec2.internal\": node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.285892 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.285813 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.344356 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.344321 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal"] Jul 08 21:32:57.344472 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.344400 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.345976 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.345960 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.346042 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.345990 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.346042 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.346004 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.348413 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.348400 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.348539 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.348511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.348578 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.348557 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.349085 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.349070 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.349085 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.349076 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.349204 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.349097 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.349204 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.349111 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.349204 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.349100 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.349298 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.349217 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.351368 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.351355 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.351409 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.351379 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.353018 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.353003 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.353085 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.353036 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.353085 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.353046 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.381952 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.381924 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-8.ec2.internal\" not found" node="ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.386446 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.386427 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.386531 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.386508 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-8.ec2.internal\" not found" node="ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.396769 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.396750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff34105d0dcd17ef37ae6f670566787e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-8.ec2.internal\" (UID: \"ff34105d0dcd17ef37ae6f670566787e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.396853 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.396779 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cd3d8d68fc7acd1f71e492925e57a6f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"6cd3d8d68fc7acd1f71e492925e57a6f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.396853 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.396797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cd3d8d68fc7acd1f71e492925e57a6f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"6cd3d8d68fc7acd1f71e492925e57a6f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.486542 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.486499 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.497887 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.497860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff34105d0dcd17ef37ae6f670566787e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-8.ec2.internal\" (UID: \"ff34105d0dcd17ef37ae6f670566787e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.497989 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.497893 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cd3d8d68fc7acd1f71e492925e57a6f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"6cd3d8d68fc7acd1f71e492925e57a6f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.497989 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.497910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cd3d8d68fc7acd1f71e492925e57a6f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"6cd3d8d68fc7acd1f71e492925e57a6f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.497989 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.497961 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cd3d8d68fc7acd1f71e492925e57a6f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"6cd3d8d68fc7acd1f71e492925e57a6f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.497989 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.497959 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff34105d0dcd17ef37ae6f670566787e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-8.ec2.internal\" (UID: \"ff34105d0dcd17ef37ae6f670566787e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.498120 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.497989 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cd3d8d68fc7acd1f71e492925e57a6f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"6cd3d8d68fc7acd1f71e492925e57a6f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.587647 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.587563 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.685317 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.685273 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.688430 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.688407 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.690602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:57.690588 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jul 08 21:32:57.789641 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.789461 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.890047 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.889962 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:57.990720 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:57.990689 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:58.035477 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.035452 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 21:32:58.035829 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.035600 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:32:58.075449 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.075421 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:58.091345 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:58.091301 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:58.124895 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.124837 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 21:27:57 +0000 UTC" deadline="2027-12-19 20:01:03.692798214 +0000 UTC" Jul 08 21:32:58.124895 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.124890 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12694h28m5.567914548s" Jul 08 21:32:58.148457 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.148371 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:58.149963 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.149937 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:58.150096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.149975 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:58.150096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.149987 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:58.192385 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:58.192351 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:58.249244 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:58.249210 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff34105d0dcd17ef37ae6f670566787e.slice/crio-cc1965c504193db779abb945c7b52a110d39bef372ae5213d3136ba75b8c69cb WatchSource:0}: Error finding container cc1965c504193db779abb945c7b52a110d39bef372ae5213d3136ba75b8c69cb: Status 404 returned error can't find the container with id cc1965c504193db779abb945c7b52a110d39bef372ae5213d3136ba75b8c69cb Jul 08 21:32:58.249784 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:32:58.249758 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd3d8d68fc7acd1f71e492925e57a6f.slice/crio-ae318d2ed757893512dd71b595078c7930a25c45df3ed1238ff8ab2dcb2f34f7 WatchSource:0}: Error finding container ae318d2ed757893512dd71b595078c7930a25c45df3ed1238ff8ab2dcb2f34f7: Status 404 returned error can't find the container with id ae318d2ed757893512dd71b595078c7930a25c45df3ed1238ff8ab2dcb2f34f7 Jul 08 21:32:58.254281 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.254257 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:32:58.292914 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:58.292876 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:58.393723 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:58.393692 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jul 08 21:32:58.469628 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.469553 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:58.471090 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.471071 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:58.494440 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.494390 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jul 08 21:32:58.508419 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.508397 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:32:58.509302 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.509289 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jul 08 21:32:58.521030 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:58.521006 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:32:59.075354 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.075312 2563 apiserver.go:52] "Watching apiserver" Jul 08 21:32:59.083530 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.083504 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 21:32:59.083927 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.083904 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal","openshift-multus/multus-4f6wl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5","openshift-multus/multus-additional-cni-plugins-qh4v8","openshift-multus/network-metrics-daemon-zqhjt","openshift-network-diagnostics/network-check-target-tvbcv","openshift-network-operator/iptables-alerter-924zv","openshift-ovn-kubernetes/ovnkube-node-5jvqs","kube-system/global-pull-secret-syncer-hqqrf","kube-system/konnectivity-agent-zvbhb","kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bkmc6","openshift-dns/node-resolver-fgr7d","openshift-image-registry/node-ca-ddhjc"] Jul 08 21:32:59.086657 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.086635 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.089346 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.088957 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.089866 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.089731 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5rlrx\"" Jul 08 21:32:59.089866 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.089744 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.089866 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.089759 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.089866 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.089747 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 21:32:59.091167 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.091124 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.091291 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.091189 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8z5ch\"" Jul 08 21:32:59.091872 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.091489 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.091872 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.091864 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.093566 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.093530 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.093701 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.093564 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.093701 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.093590 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 21:32:59.093824 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.093766 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qwrzf\"" Jul 08 21:32:59.093920 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.093905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.094439 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.094407 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 21:32:59.096190 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096169 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.096293 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.096249 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:32:59.096415 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096392 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 21:32:59.096524 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096512 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v4tt4\"" Jul 08 21:32:59.096648 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096634 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.096788 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096775 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.096852 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096840 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 21:32:59.096916 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.096904 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 21:32:59.098721 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.098702 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:32:59.098815 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.098770 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:32:59.101173 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.101153 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.101271 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.101220 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:32:59.103523 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.103503 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.105940 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.105913 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.106143 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed16c060-6133-43f3-9eb5-5bbc58e352f0-hosts-file\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.106272 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed16c060-6133-43f3-9eb5-5bbc58e352f0-tmp-dir\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.106272 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106272 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106224 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-device-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106272 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106247 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106294 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cnibin\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.106492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlxj4\" (UniqueName: \"kubernetes.io/projected/ed16c060-6133-43f3-9eb5-5bbc58e352f0-kube-api-access-zlxj4\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.106492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106377 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-socket-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-registration-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-os-release\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.106492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106469 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106495 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9abdf550-5e2b-4f07-995e-c0aa60054089-iptables-alerter-script\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-sys-fs\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t55dd\" (UniqueName: \"kubernetes.io/projected/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-kube-api-access-t55dd\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106644 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cni-binary-copy\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106685 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r7fx\" (UniqueName: \"kubernetes.io/projected/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-kube-api-access-4r7fx\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.106754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.106732 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9abdf550-5e2b-4f07-995e-c0aa60054089-host-slash\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.107145 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.107048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrdb\" (UniqueName: \"kubernetes.io/projected/9abdf550-5e2b-4f07-995e-c0aa60054089-kube-api-access-vwrdb\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.107145 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.107116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-system-cni-dir\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.107280 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.107164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9vss\" (UniqueName: \"kubernetes.io/projected/6bfb4fef-2218-465f-95a6-8515c5c7c77a-kube-api-access-x9vss\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.108468 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.108446 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8k4vg\"" Jul 08 21:32:59.108572 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.108558 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 21:32:59.108634 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.108604 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.108688 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.108655 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.109096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.109074 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 21:32:59.109517 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.109343 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.111497 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.111350 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n9jc6\"" Jul 08 21:32:59.112065 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.112025 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 21:32:59.112435 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.112418 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 21:32:59.112668 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.112640 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gmr52\"" Jul 08 21:32:59.112975 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.112956 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.114370 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.114351 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:32:59.115402 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.115149 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 21:32:59.115402 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.115158 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 21:32:59.116129 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.116107 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 21:32:59.116244 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.116183 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 21:32:59.116377 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.116358 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.116650 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.116628 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.116805 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.116781 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.116889 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.116864 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ctc4r\"" Jul 08 21:32:59.119227 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.119206 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.119441 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.119424 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.119717 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.119455 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6fq6d\"" Jul 08 21:32:59.134057 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.134035 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v5p2r" Jul 08 21:32:59.142698 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.142677 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v5p2r" Jul 08 21:32:59.195478 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.195448 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 21:32:59.207446 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207412 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-sys-fs\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.207639 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207457 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r7fx\" (UniqueName: \"kubernetes.io/projected/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-kube-api-access-4r7fx\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.207639 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-kubelet-config\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.207639 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-cnibin\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.207639 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207556 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-sys-fs\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.207639 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-etc-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-cni-bin\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-cni-netd\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207728 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2cs\" (UniqueName: \"kubernetes.io/projected/277bf2fe-000e-401c-948f-92651939b6ed-kube-api-access-9h2cs\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9abdf550-5e2b-4f07-995e-c0aa60054089-host-slash\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f67521e-4089-4ab3-8923-40ad7bc1d024-cni-binary-copy\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207804 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-cni-multus\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzfg7\" (UniqueName: \"kubernetes.io/projected/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-kube-api-access-vzfg7\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207843 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9abdf550-5e2b-4f07-995e-c0aa60054089-host-slash\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.207877 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207867 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp4bp\" (UniqueName: \"kubernetes.io/projected/7378a310-c9e1-4251-9d7d-aad8a15aac4a-kube-api-access-jp4bp\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207954 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed16c060-6133-43f3-9eb5-5bbc58e352f0-hosts-file\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207980 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-device-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.207996 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed16c060-6133-43f3-9eb5-5bbc58e352f0-hosts-file\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cnibin\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208042 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-device-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9vss\" (UniqueName: \"kubernetes.io/projected/6bfb4fef-2218-465f-95a6-8515c5c7c77a-kube-api-access-x9vss\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlxj4\" (UniqueName: \"kubernetes.io/projected/ed16c060-6133-43f3-9eb5-5bbc58e352f0-kube-api-access-zlxj4\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208127 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208143 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-registration-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208234 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-registration-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cnibin\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208323 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-conf-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.208400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208397 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-serviceca\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208423 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-log-socket\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208447 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/277bf2fe-000e-401c-948f-92651939b6ed-etc-tuned\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t55dd\" (UniqueName: \"kubernetes.io/projected/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-kube-api-access-t55dd\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208534 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208554 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-system-cni-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208615 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-socket-dir-parent\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208642 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-multus-certs\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208677 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-systemd-units\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.208715 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-sys\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.208794 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:32:59.708771169 +0000 UTC m=+2.930781370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208813 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-cni-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-cni-bin\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208856 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-ovn\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysctl-d\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.209401 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-run\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed16c060-6133-43f3-9eb5-5bbc58e352f0-tmp-dir\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208974 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-os-release\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.208999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-kubernetes\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-host\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209027 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209043 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-etc-kubernetes\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209067 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-systemd\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysconfig\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cni-binary-copy\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-hostroot\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209233 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed16c060-6133-43f3-9eb5-5bbc58e352f0-tmp-dir\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209235 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e23b995-db34-4684-8653-ecd69281486a-agent-certs\") pod \"konnectivity-agent-zvbhb\" (UID: \"6e23b995-db34-4684-8653-ecd69281486a\") " pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrdb\" (UniqueName: \"kubernetes.io/projected/9abdf550-5e2b-4f07-995e-c0aa60054089-kube-api-access-vwrdb\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209374 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-system-cni-dir\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.210286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209396 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-daemon-config\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-var-lib-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/277bf2fe-000e-401c-948f-92651939b6ed-tmp\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209505 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-node-log\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovnkube-config\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-lib-modules\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209566 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-var-lib-kubelet\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-socket-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-os-release\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209625 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-dbus\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-slash\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209647 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-system-cni-dir\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovn-node-metrics-cert\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209769 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cni-binary-copy\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209803 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-socket-dir\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209821 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6bfb4fef-2218-465f-95a6-8515c5c7c77a-os-release\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209856 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-systemd\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209885 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9abdf550-5e2b-4f07-995e-c0aa60054089-iptables-alerter-script\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209927 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-netns\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-host\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.209994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e23b995-db34-4684-8653-ecd69281486a-konnectivity-ca\") pod \"konnectivity-agent-zvbhb\" (UID: \"6e23b995-db34-4684-8653-ecd69281486a\") " pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovnkube-script-lib\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-k8s-cni-cncf-io\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-kubelet\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbgkf\" (UniqueName: \"kubernetes.io/projected/2f67521e-4089-4ab3-8923-40ad7bc1d024-kube-api-access-kbgkf\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-kubelet\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-env-overrides\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210256 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-modprobe-d\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysctl-conf\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.211745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210305 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-run-netns\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.212242 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210303 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.212242 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210360 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9abdf550-5e2b-4f07-995e-c0aa60054089-iptables-alerter-script\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.212242 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210384 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.212242 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.210745 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6bfb4fef-2218-465f-95a6-8515c5c7c77a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.213856 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.213829 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 21:32:59.217939 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.217914 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r7fx\" (UniqueName: \"kubernetes.io/projected/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-kube-api-access-4r7fx\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.219709 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.219684 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9vss\" (UniqueName: \"kubernetes.io/projected/6bfb4fef-2218-465f-95a6-8515c5c7c77a-kube-api-access-x9vss\") pod \"multus-additional-cni-plugins-qh4v8\" (UID: \"6bfb4fef-2218-465f-95a6-8515c5c7c77a\") " pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.222882 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.222853 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlxj4\" (UniqueName: \"kubernetes.io/projected/ed16c060-6133-43f3-9eb5-5bbc58e352f0-kube-api-access-zlxj4\") pod \"node-resolver-fgr7d\" (UID: \"ed16c060-6133-43f3-9eb5-5bbc58e352f0\") " pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.223278 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.223260 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t55dd\" (UniqueName: \"kubernetes.io/projected/158974ce-d54c-4b1e-bdaf-a5e09ec7786c-kube-api-access-t55dd\") pod \"aws-ebs-csi-driver-node-nb2n5\" (UID: \"158974ce-d54c-4b1e-bdaf-a5e09ec7786c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.223727 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.223711 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrdb\" (UniqueName: \"kubernetes.io/projected/9abdf550-5e2b-4f07-995e-c0aa60054089-kube-api-access-vwrdb\") pod \"iptables-alerter-924zv\" (UID: \"9abdf550-5e2b-4f07-995e-c0aa60054089\") " pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.254174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.254098 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" event={"ID":"6cd3d8d68fc7acd1f71e492925e57a6f","Type":"ContainerStarted","Data":"ae318d2ed757893512dd71b595078c7930a25c45df3ed1238ff8ab2dcb2f34f7"} Jul 08 21:32:59.255293 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.255263 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" event={"ID":"ff34105d0dcd17ef37ae6f670566787e","Type":"ContainerStarted","Data":"cc1965c504193db779abb945c7b52a110d39bef372ae5213d3136ba75b8c69cb"} Jul 08 21:32:59.311340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.311340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311323 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-node-log\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.311340 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311349 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovnkube-config\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311369 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-lib-modules\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-var-lib-kubelet\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311435 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-node-log\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-dbus\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311490 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-var-lib-kubelet\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.311473 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311507 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-slash\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.311543 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:32:59.81152784 +0000 UTC m=+3.033538046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovn-node-metrics-cert\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311563 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-slash\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-systemd\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311604 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-systemd\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-netns\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.311632 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311625 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-host\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e23b995-db34-4684-8653-ecd69281486a-konnectivity-ca\") pod \"konnectivity-agent-zvbhb\" (UID: \"6e23b995-db34-4684-8653-ecd69281486a\") " pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311675 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovnkube-script-lib\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311691 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-k8s-cni-cncf-io\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-kubelet\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbgkf\" (UniqueName: \"kubernetes.io/projected/2f67521e-4089-4ab3-8923-40ad7bc1d024-kube-api-access-kbgkf\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311699 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-dbus\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-kubelet\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-env-overrides\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-modprobe-d\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311797 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-netns\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysctl-conf\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311827 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-host\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311831 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-run-netns\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311849 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-kubelet-config\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-cnibin\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-etc-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312361 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311941 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-cni-bin\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-cni-netd\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311981 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2cs\" (UniqueName: \"kubernetes.io/projected/277bf2fe-000e-401c-948f-92651939b6ed-kube-api-access-9h2cs\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.311999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f67521e-4089-4ab3-8923-40ad7bc1d024-cni-binary-copy\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-cni-multus\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzfg7\" (UniqueName: \"kubernetes.io/projected/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-kube-api-access-vzfg7\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312052 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp4bp\" (UniqueName: \"kubernetes.io/projected/7378a310-c9e1-4251-9d7d-aad8a15aac4a-kube-api-access-jp4bp\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312069 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovnkube-config\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312176 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-cni-bin\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312184 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-kubelet-config\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312232 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-etc-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-lib-modules\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312243 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-cni-netd\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312302 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e23b995-db34-4684-8653-ecd69281486a-konnectivity-ca\") pod \"konnectivity-agent-zvbhb\" (UID: \"6e23b995-db34-4684-8653-ecd69281486a\") " pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312317 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.312957 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312316 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-cni-multus\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312495 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-run-netns\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312611 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-modprobe-d\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312672 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-conf-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-serviceca\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-log-socket\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312773 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/277bf2fe-000e-401c-948f-92651939b6ed-etc-tuned\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312793 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysctl-conf\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-system-cni-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-socket-dir-parent\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-multus-certs\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-systemd-units\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312931 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-sys\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-cni-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312966 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f67521e-4089-4ab3-8923-40ad7bc1d024-cni-binary-copy\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.312983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-cni-bin\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313008 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-ovn\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.313516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313039 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-kubelet\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysctl-d\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-run\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313082 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovnkube-script-lib\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313097 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-log-socket\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313106 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-os-release\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313119 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-conf-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-kubernetes\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313158 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-system-cni-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313158 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-host\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313221 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-host\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-etc-kubernetes\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313262 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-systemd\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313011 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-k8s-cni-cncf-io\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysconfig\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313318 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-run-multus-certs\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-hostroot\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e23b995-db34-4684-8653-ecd69281486a-agent-certs\") pod \"konnectivity-agent-zvbhb\" (UID: \"6e23b995-db34-4684-8653-ecd69281486a\") " pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.314607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313364 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-host-var-lib-cni-bin\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313405 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-ovn\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313416 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-cni-dir\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313419 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-daemon-config\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-systemd-units\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313462 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-var-lib-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313495 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/277bf2fe-000e-401c-948f-92651939b6ed-tmp\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313499 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-serviceca\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313075 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-socket-dir-parent\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-sys\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313680 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-os-release\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313718 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-systemd\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313823 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysctl-d\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313861 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-run\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313896 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7378a310-c9e1-4251-9d7d-aad8a15aac4a-env-overrides\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313906 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-host-kubelet\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313929 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2f67521e-4089-4ab3-8923-40ad7bc1d024-multus-daemon-config\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.315561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313955 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-kubernetes\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313964 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-hostroot\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313253 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-etc-kubernetes\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.314007 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-var-lib-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.314019 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/277bf2fe-000e-401c-948f-92651939b6ed-etc-sysconfig\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.313087 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f67521e-4089-4ab3-8923-40ad7bc1d024-cnibin\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.314045 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7378a310-c9e1-4251-9d7d-aad8a15aac4a-run-openvswitch\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.314973 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7378a310-c9e1-4251-9d7d-aad8a15aac4a-ovn-node-metrics-cert\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.316164 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/277bf2fe-000e-401c-948f-92651939b6ed-etc-tuned\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.316465 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.316386 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/277bf2fe-000e-401c-948f-92651939b6ed-tmp\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.317267 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.317203 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e23b995-db34-4684-8653-ecd69281486a-agent-certs\") pod \"konnectivity-agent-zvbhb\" (UID: \"6e23b995-db34-4684-8653-ecd69281486a\") " pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.323022 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.322996 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:32:59.323261 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.323027 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:32:59.323261 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.323043 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.323261 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.323115 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:32:59.82309315 +0000 UTC m=+3.045103362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.328223 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.328146 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzfg7\" (UniqueName: \"kubernetes.io/projected/e6304ba3-31c3-46c9-aa0a-0d9397cbdd27-kube-api-access-vzfg7\") pod \"node-ca-ddhjc\" (UID: \"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27\") " pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.329624 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.329587 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp4bp\" (UniqueName: \"kubernetes.io/projected/7378a310-c9e1-4251-9d7d-aad8a15aac4a-kube-api-access-jp4bp\") pod \"ovnkube-node-5jvqs\" (UID: \"7378a310-c9e1-4251-9d7d-aad8a15aac4a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.329842 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.329818 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbgkf\" (UniqueName: \"kubernetes.io/projected/2f67521e-4089-4ab3-8923-40ad7bc1d024-kube-api-access-kbgkf\") pod \"multus-4f6wl\" (UID: \"2f67521e-4089-4ab3-8923-40ad7bc1d024\") " pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.330156 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.330132 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2cs\" (UniqueName: \"kubernetes.io/projected/277bf2fe-000e-401c-948f-92651939b6ed-kube-api-access-9h2cs\") pod \"tuned-bkmc6\" (UID: \"277bf2fe-000e-401c-948f-92651939b6ed\") " pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.387576 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.387514 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:59.399773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.399739 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-924zv" Jul 08 21:32:59.408083 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.408042 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fgr7d" Jul 08 21:32:59.422347 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.422323 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" Jul 08 21:32:59.428301 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.428278 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" Jul 08 21:32:59.436287 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.436261 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ddhjc" Jul 08 21:32:59.444182 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.444162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4f6wl" Jul 08 21:32:59.453184 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.453155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:32:59.461364 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.461345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:32:59.467359 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.467336 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" Jul 08 21:32:59.716602 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.716489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:32:59.716742 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.716653 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.716742 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.716728 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:33:00.716706081 +0000 UTC m=+3.938716282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.816937 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.816904 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:32:59.817114 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.817087 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.817189 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.817170 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:00.817152457 +0000 UTC m=+4.039162654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.917494 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:32:59.917454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:32:59.917666 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.917645 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:32:59.917770 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.917674 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:32:59.917770 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.917691 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.917770 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:32:59.917756 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:33:00.917735346 +0000 UTC m=+4.139745549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:00.146195 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.146110 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:27:59 +0000 UTC" deadline="2028-02-17 21:45:57.284788307 +0000 UTC" Jul 08 21:33:00.146195 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.146189 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14136h12m57.138604347s" Jul 08 21:33:00.245379 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.245343 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:00.245559 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.245463 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:00.245559 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.245473 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:00.245559 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.245503 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:00.245714 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.245626 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:00.245754 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.245719 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:00.364285 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:00.364246 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e23b995_db34_4684_8653_ecd69281486a.slice/crio-70943d4d55b354c31de1eea6d8aa3dc21133787183ee92d5fb06f1d4ac150928 WatchSource:0}: Error finding container 70943d4d55b354c31de1eea6d8aa3dc21133787183ee92d5fb06f1d4ac150928: Status 404 returned error can't find the container with id 70943d4d55b354c31de1eea6d8aa3dc21133787183ee92d5fb06f1d4ac150928 Jul 08 21:33:00.724107 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.723881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:00.724311 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.724051 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:00.724311 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.724262 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:33:02.724243817 +0000 UTC m=+5.946253997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:00.825006 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.824951 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:00.825207 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.825134 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:00.825262 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.825232 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:02.825211024 +0000 UTC m=+6.047221223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:00.926541 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:00.926406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:00.926723 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.926619 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:00.926723 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.926643 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:00.926723 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.926657 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:00.926723 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:00.926721 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:33:02.92670064 +0000 UTC m=+6.148710829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:01.147276 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.147155 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:27:59 +0000 UTC" deadline="2028-01-29 15:03:21.377345862 +0000 UTC" Jul 08 21:33:01.147276 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.147241 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13673h30m20.230109524s" Jul 08 21:33:01.284702 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.284609 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerStarted","Data":"3a0bdb08f99ca3cf1e41676b4ad37dcbf337378b453ae56aa6ba8c523de06e1c"} Jul 08 21:33:01.289112 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.289072 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4f6wl" event={"ID":"2f67521e-4089-4ab3-8923-40ad7bc1d024","Type":"ContainerStarted","Data":"a4bd9842c85bb95b703455136e00eba303e0ecc3d359c353f2d96ce383eb0c2f"} Jul 08 21:33:01.302505 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.302369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"c938f7f158e75e29f6720e5f92ef9071516b74c62e2a5c11102d5da2e8e9230d"} Jul 08 21:33:01.308533 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.308491 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ddhjc" event={"ID":"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27","Type":"ContainerStarted","Data":"9c87719f13d60656efc1d5304e38a1d46385568701c1d2ac8bb1b94f1f39f557"} Jul 08 21:33:01.330710 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.330667 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zvbhb" event={"ID":"6e23b995-db34-4684-8653-ecd69281486a","Type":"ContainerStarted","Data":"70943d4d55b354c31de1eea6d8aa3dc21133787183ee92d5fb06f1d4ac150928"} Jul 08 21:33:01.339728 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.339685 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" event={"ID":"158974ce-d54c-4b1e-bdaf-a5e09ec7786c","Type":"ContainerStarted","Data":"48bac214882ac0f7d84c6544a5dae7c329e846d1ee16f2e42d9d9e88880b9f60"} Jul 08 21:33:01.350607 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.350529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fgr7d" event={"ID":"ed16c060-6133-43f3-9eb5-5bbc58e352f0","Type":"ContainerStarted","Data":"420cb332bf05f5999b17c875485be0e648a19249237f8120980a7682aca87dc8"} Jul 08 21:33:01.359474 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.359434 2563 generic.go:356] "Generic (PLEG): container finished" podID="6cd3d8d68fc7acd1f71e492925e57a6f" containerID="f2298a527c4c7bfd0e2128110e1c83e1c4de698587b6a68984382baf7e612c5b" exitCode=0 Jul 08 21:33:01.359644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.359626 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" event={"ID":"6cd3d8d68fc7acd1f71e492925e57a6f","Type":"ContainerDied","Data":"f2298a527c4c7bfd0e2128110e1c83e1c4de698587b6a68984382baf7e612c5b"} Jul 08 21:33:01.377852 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.377807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" event={"ID":"ff34105d0dcd17ef37ae6f670566787e","Type":"ContainerStarted","Data":"168080394b61dcade4d77acb263d7022c5defd5e9dac2875cc93b8af40c6da6b"} Jul 08 21:33:01.396561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.396517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-924zv" event={"ID":"9abdf550-5e2b-4f07-995e-c0aa60054089","Type":"ContainerStarted","Data":"b225154578b4d016ebefd1cae80169d95f7a74e91c13c7929d458bfbd937de94"} Jul 08 21:33:01.416645 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:01.416605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" event={"ID":"277bf2fe-000e-401c-948f-92651939b6ed","Type":"ContainerStarted","Data":"dfaba9ff058ee1d6996d107f0390b771c9dbf333fb0e6fae580ee19f6901d4ae"} Jul 08 21:33:02.245420 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.244453 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:02.245420 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.244617 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:02.245420 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.244773 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:02.245420 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.244875 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:02.245420 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.244968 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:02.245420 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.245054 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:02.442231 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.439400 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" event={"ID":"6cd3d8d68fc7acd1f71e492925e57a6f","Type":"ContainerStarted","Data":"cfa463f27af65f0b17cea484287dba766b46b7f9bc525447f1619a0740da1de7"} Jul 08 21:33:02.457678 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.457599 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" podStartSLOduration=3.457574153 podStartE2EDuration="3.457574153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:02.457544862 +0000 UTC m=+5.679555064" watchObservedRunningTime="2026-07-08 21:33:02.457574153 +0000 UTC m=+5.679584361" Jul 08 21:33:02.458650 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.458597 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" podStartSLOduration=3.4585777220000002 podStartE2EDuration="3.458577722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:01.39296402 +0000 UTC m=+4.614974257" watchObservedRunningTime="2026-07-08 21:33:02.458577722 +0000 UTC m=+5.680587930" Jul 08 21:33:02.743984 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.743568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:02.743984 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.743761 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:02.743984 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.743849 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:33:06.743825513 +0000 UTC m=+9.965835713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:02.844597 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.844551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:02.844803 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.844788 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:02.844871 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.844861 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:06.844841564 +0000 UTC m=+10.066851752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:02.946229 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:02.946186 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:02.946450 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.946431 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:02.946516 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.946460 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:02.946516 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.946474 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:02.946622 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:02.946541 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:33:06.946519922 +0000 UTC m=+10.168530125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:04.244470 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:04.244427 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:04.244470 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:04.244462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:04.245000 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:04.244560 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:04.245000 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:04.244427 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:04.245000 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:04.244689 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:04.245000 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:04.244778 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:06.245346 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:06.245308 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:06.245848 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.245469 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:06.245935 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:06.245867 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:06.245991 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.245959 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:06.246047 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:06.246011 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:06.246104 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.246087 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:06.776376 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:06.776336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:06.776596 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.776520 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:06.776596 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.776591 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:33:14.776569956 +0000 UTC m=+17.998580143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:06.877299 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:06.877259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:06.877506 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.877475 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:06.877568 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.877545 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:14.877525364 +0000 UTC m=+18.099535571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:06.978011 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:06.977970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:06.978643 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.978233 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:06.978643 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.978265 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:06.978643 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.978281 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:06.978643 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:06.978338 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:33:14.978320929 +0000 UTC m=+18.200331127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:08.244686 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:08.244636 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:08.245218 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:08.244648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:08.245218 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:08.244782 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:08.245218 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:08.244655 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:08.245218 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:08.244844 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:08.245218 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:08.244988 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:10.244673 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:10.244640 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:10.245229 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:10.244640 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:10.245229 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:10.244758 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:10.245229 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:10.244847 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:10.245229 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:10.244640 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:10.245229 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:10.244941 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:12.245084 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:12.245052 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:12.245572 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:12.245057 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:12.245572 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:12.245190 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:12.245572 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:12.245209 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:12.245572 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:12.245299 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:12.245572 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:12.245397 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:14.244896 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:14.244858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:14.245437 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:14.244858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:14.245437 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.244971 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:14.245437 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:14.244858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:14.245437 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.245067 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:14.245437 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.245202 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:14.844397 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:14.844365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:14.844552 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.844536 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:14.844619 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.844606 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:33:30.8445841 +0000 UTC m=+34.066594279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:14.945394 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:14.945359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:14.945563 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.945492 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:14.945563 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:14.945555 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:30.945542006 +0000 UTC m=+34.167552186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:15.046369 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:15.046334 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:15.046553 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:15.046532 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:15.046593 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:15.046560 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:15.046593 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:15.046574 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:15.046657 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:15.046637 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:33:31.046617251 +0000 UTC m=+34.268627435 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:16.245172 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:16.245117 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:16.245625 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:16.245148 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:16.245625 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:16.245265 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:16.245625 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:16.245279 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:16.245625 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:16.245396 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:16.245625 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:16.245461 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:18.244668 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:18.244631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:18.244668 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:18.244664 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:18.245151 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:18.244631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:18.245151 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:18.244782 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:18.245245 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:18.245216 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:18.245288 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:18.245270 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:20.245409 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.245350 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:20.245409 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.245369 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:20.245409 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.245386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:20.246077 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:20.245485 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:20.246077 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:20.245620 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:20.246077 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:20.245712 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:20.485166 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.485107 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" event={"ID":"277bf2fe-000e-401c-948f-92651939b6ed","Type":"ContainerStarted","Data":"04fd741e48a9f17348227cd3ee555ab99370cd98c8cc233a165a4e6a9e09f08c"} Jul 08 21:33:20.486571 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.486547 2563 generic.go:356] "Generic (PLEG): container finished" podID="6bfb4fef-2218-465f-95a6-8515c5c7c77a" containerID="d257a20233c919901299873fdf69780a584bab939cbabc676de49ce4d04612b4" exitCode=0 Jul 08 21:33:20.486688 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.486606 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerDied","Data":"d257a20233c919901299873fdf69780a584bab939cbabc676de49ce4d04612b4"} Jul 08 21:33:20.488207 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.488180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4f6wl" event={"ID":"2f67521e-4089-4ab3-8923-40ad7bc1d024","Type":"ContainerStarted","Data":"52ec784fdd16758e1ffbea1500c81cfeb4b29c30da1864343d730bd0a67f2c3b"} Jul 08 21:33:20.491375 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.491204 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"03894916be3725f853047e7467921a341b97a8c50b1eca621ec318d7988b514c"} Jul 08 21:33:20.491485 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.491452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"31a5b9b28cf484ef4798236e8cfafbee1ea67a022efb17b4ad76617a86372e68"} Jul 08 21:33:20.491537 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.491488 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"a5d03056ecf044b5d860a1815c81a7677c6c869af4d0f4af7378dc55182405bc"} Jul 08 21:33:20.491537 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.491517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"6651b226d8f270c46de2dd98939b89e4e3c9dd58db5180712b149a201962e520"} Jul 08 21:33:20.491606 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.491547 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"f29b1c16bfd53d46cf2646a66856816410d3def19914776b087de1b8becf4f3b"} Jul 08 21:33:20.491606 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.491574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"1a9d7e8ce921284472659c2571cb4b546a1c3cc11530058deb0bbf8966ad1236"} Jul 08 21:33:20.493198 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.493051 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ddhjc" event={"ID":"e6304ba3-31c3-46c9-aa0a-0d9397cbdd27","Type":"ContainerStarted","Data":"45d6cdac254198e5b455771e42dd77e365f6f0b82efcdb7d6bd1337b12dc5e4e"} Jul 08 21:33:20.494575 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.494546 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zvbhb" event={"ID":"6e23b995-db34-4684-8653-ecd69281486a","Type":"ContainerStarted","Data":"a04e9a2f59ce51a3de1ee2d2b608b934a8e44c2368460a8d310e00a50b1219b3"} Jul 08 21:33:20.496221 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.496192 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" event={"ID":"158974ce-d54c-4b1e-bdaf-a5e09ec7786c","Type":"ContainerStarted","Data":"82e8471183f41106251e23283e4d2986deddd624edca8ad26885f6d4b2c605ed"} Jul 08 21:33:20.497550 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.497532 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fgr7d" event={"ID":"ed16c060-6133-43f3-9eb5-5bbc58e352f0","Type":"ContainerStarted","Data":"77976e5fb1627c5fad23cf719d883989c24423318ff1d0f1dc1843e024a5949f"} Jul 08 21:33:20.522010 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.521966 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bkmc6" podStartSLOduration=3.004307347 podStartE2EDuration="22.52194997s" totalImagesPullingTime="19.517642623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.502818203 +0000 UTC m=+23.724828401" watchObservedRunningTime="2026-07-08 21:33:20.52194997 +0000 UTC m=+23.743960187" Jul 08 21:33:20.522163 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.522085 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ddhjc" podStartSLOduration=3.001908043 podStartE2EDuration="22.522080721s" totalImagesPullingTime="19.520172678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.522017552 +0000 UTC m=+23.744027755" watchObservedRunningTime="2026-07-08 21:33:20.522080721 +0000 UTC m=+23.744090922" Jul 08 21:33:20.537006 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.536953 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zvbhb" podStartSLOduration=3.015474305 podStartE2EDuration="22.536938627s" totalImagesPullingTime="19.521464322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.536878272 +0000 UTC m=+23.758888472" watchObservedRunningTime="2026-07-08 21:33:20.536938627 +0000 UTC m=+23.758948828" Jul 08 21:33:20.552509 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.552460 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fgr7d" podStartSLOduration=3.026807872 podStartE2EDuration="22.552445479s" totalImagesPullingTime="19.525637607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.552155986 +0000 UTC m=+23.774166184" watchObservedRunningTime="2026-07-08 21:33:20.552445479 +0000 UTC m=+23.774455679" Jul 08 21:33:20.591625 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:20.591579 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4f6wl" podStartSLOduration=3.020842662 podStartE2EDuration="22.59156462s" totalImagesPullingTime="19.570721958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.590956442 +0000 UTC m=+23.812966644" watchObservedRunningTime="2026-07-08 21:33:20.59156462 +0000 UTC m=+23.813574821" Jul 08 21:33:21.501343 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:21.501299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-924zv" event={"ID":"9abdf550-5e2b-4f07-995e-c0aa60054089","Type":"ContainerStarted","Data":"e467f522fd3f245764e993c33dde49a55485df2e781be07d258b487b61f78da6"} Jul 08 21:33:21.986634 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:21.986605 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 21:33:22.161609 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.161506 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T21:33:21.986622745Z","UUID":"9b32e977-f178-4a73-b5ef-fb73816ed61f","Handler":null,"Name":"","Endpoint":""} Jul 08 21:33:22.162950 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.162930 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 21:33:22.163068 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.162967 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 21:33:22.244876 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.244839 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:22.244876 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.244872 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:22.245086 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.244872 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:22.245086 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:22.244976 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:22.245086 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:22.245076 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:22.245222 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:22.245166 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:22.507138 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:22.507035 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" event={"ID":"158974ce-d54c-4b1e-bdaf-a5e09ec7786c","Type":"ContainerStarted","Data":"debf797752e947eff04a780e83f0e8e6906d6c87626148c82cebbae3e8016f6c"} Jul 08 21:33:23.512698 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:23.512661 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"8e1a3e28e96c0ebc9ec53a9fb0efb8ff5ed576d3c180a85ccc9e52b082ed46cf"} Jul 08 21:33:23.514654 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:23.514626 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" event={"ID":"158974ce-d54c-4b1e-bdaf-a5e09ec7786c","Type":"ContainerStarted","Data":"3233c04ec2aa95f2fed6661641a3dff456697e9782b99cb94e9f37fe87171870"} Jul 08 21:33:23.533331 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:23.533254 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nb2n5" podStartSLOduration=2.999689611 podStartE2EDuration="25.533237294s" totalImagesPullingTime="22.533547683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:23.533066604 +0000 UTC m=+26.755076804" watchObservedRunningTime="2026-07-08 21:33:23.533237294 +0000 UTC m=+26.755247498" Jul 08 21:33:23.534218 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:23.534161 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-924zv" podStartSLOduration=6.017709089 podStartE2EDuration="25.534143167s" totalImagesPullingTime="19.516434078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:21.531857492 +0000 UTC m=+24.753867693" watchObservedRunningTime="2026-07-08 21:33:23.534143167 +0000 UTC m=+26.756153367" Jul 08 21:33:23.638266 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:23.638238 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:33:23.638917 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:23.638892 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:33:24.244526 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:24.244492 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:24.244699 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:24.244492 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:24.244699 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:24.244600 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:24.244699 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:24.244492 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:24.244813 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:24.244663 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:24.244855 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:24.244788 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:24.517713 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:24.517628 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:33:24.518099 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:24.518000 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zvbhb" Jul 08 21:33:26.245472 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.245348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:26.245906 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.245373 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:26.245906 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:26.245558 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:26.245906 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:26.245619 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:26.245906 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.245394 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:26.245906 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:26.245706 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:26.523973 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.523860 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerStarted","Data":"abbba0037ed90430e20cd01118abb894885b29d151323ef9231acd44c1630e7e"} Jul 08 21:33:26.527327 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.527303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" event={"ID":"7378a310-c9e1-4251-9d7d-aad8a15aac4a","Type":"ContainerStarted","Data":"2a514123bf7bc8b41a621acb7f5b2252bf73e2cdaf52c4407cc078cbfa6f64a0"} Jul 08 21:33:26.527478 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.527453 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:33:26.527571 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.527527 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:33:26.527636 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.527577 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:33:26.548630 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.548607 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:33:26.555572 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.555551 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:33:26.610007 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:26.609963 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" podStartSLOduration=9.023677099 podStartE2EDuration="28.609948421s" totalImagesPullingTime="19.586271322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:26.581824019 +0000 UTC m=+29.803834221" watchObservedRunningTime="2026-07-08 21:33:26.609948421 +0000 UTC m=+29.831958621" Jul 08 21:33:27.533585 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:27.533551 2563 generic.go:356] "Generic (PLEG): container finished" podID="6bfb4fef-2218-465f-95a6-8515c5c7c77a" containerID="abbba0037ed90430e20cd01118abb894885b29d151323ef9231acd44c1630e7e" exitCode=0 Jul 08 21:33:27.534151 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:27.533628 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerDied","Data":"abbba0037ed90430e20cd01118abb894885b29d151323ef9231acd44c1630e7e"} Jul 08 21:33:28.244727 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.244544 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:28.244922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.244544 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:28.244922 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:28.244811 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:28.244922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.244545 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:28.244922 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:28.244877 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:28.245081 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:28.244944 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:28.330317 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.330285 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zqhjt"] Jul 08 21:33:28.333400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.333360 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hqqrf"] Jul 08 21:33:28.334218 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.334177 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tvbcv"] Jul 08 21:33:28.536387 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.536356 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:28.536795 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.536356 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:28.536795 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:28.536467 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:28.536795 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:28.536356 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:28.536795 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:28.536532 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:28.536795 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:28.536612 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:29.541021 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:29.540936 2563 generic.go:356] "Generic (PLEG): container finished" podID="6bfb4fef-2218-465f-95a6-8515c5c7c77a" containerID="27c5ac9b3d7d60f6f326c1f51889f38eb4a9ae2abcb090ba7720c6a43137053e" exitCode=0 Jul 08 21:33:29.541021 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:29.541008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerDied","Data":"27c5ac9b3d7d60f6f326c1f51889f38eb4a9ae2abcb090ba7720c6a43137053e"} Jul 08 21:33:30.244803 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.244769 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:30.244915 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.244829 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:30.244988 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.244971 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:30.245027 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.244988 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:30.245069 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.245050 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:30.245161 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.245137 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:30.545522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.545432 2563 generic.go:356] "Generic (PLEG): container finished" podID="6bfb4fef-2218-465f-95a6-8515c5c7c77a" containerID="b3cb4d883ef45180af944abe26da7b0bf5bf5f250406cbb8286cafa3bb1705b9" exitCode=0 Jul 08 21:33:30.545522 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.545497 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerDied","Data":"b3cb4d883ef45180af944abe26da7b0bf5bf5f250406cbb8286cafa3bb1705b9"} Jul 08 21:33:30.863295 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.863263 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:30.863472 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.863428 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:30.863538 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.863502 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs podName:95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c nodeName:}" failed. No retries permitted until 2026-07-08 21:34:02.863481969 +0000 UTC m=+66.085492155 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs") pod "network-metrics-daemon-zqhjt" (UID: "95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:30.963914 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:30.963883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:30.964102 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.964049 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:30.964181 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:30.964145 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret podName:1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:02.964107949 +0000 UTC m=+66.186118152 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret") pod "global-pull-secret-syncer-hqqrf" (UID: "1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:31.065262 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:31.065223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:31.065439 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:31.065402 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:31.065439 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:31.065426 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:31.065515 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:31.065440 2563 projected.go:196] Error preparing data for projected volume kube-api-access-c4whd for pod openshift-network-diagnostics/network-check-target-tvbcv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:31.065515 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:31.065497 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd podName:52e37e95-9c50-4b9b-8fb4-4cf78d479d8d nodeName:}" failed. No retries permitted until 2026-07-08 21:34:03.065480871 +0000 UTC m=+66.287491066 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c4whd" (UniqueName: "kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd") pod "network-check-target-tvbcv" (UID: "52e37e95-9c50-4b9b-8fb4-4cf78d479d8d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:32.244830 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.244789 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:32.244830 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.244809 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:32.244830 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.244809 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:32.245365 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:32.244921 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hqqrf" podUID="1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9" Jul 08 21:33:32.245365 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:32.245010 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zqhjt" podUID="95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c" Jul 08 21:33:32.245365 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:32.245102 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tvbcv" podUID="52e37e95-9c50-4b9b-8fb4-4cf78d479d8d" Jul 08 21:33:32.537392 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.537312 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeReady" Jul 08 21:33:32.537533 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.537454 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 21:33:32.623163 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.623109 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-86qgs"] Jul 08 21:33:32.627589 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.627563 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d8qct"] Jul 08 21:33:32.627733 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.627667 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.630721 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.629709 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 21:33:32.630721 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.629720 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n8f5q\"" Jul 08 21:33:32.630721 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.629710 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 21:33:32.631798 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.631780 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.634101 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.633972 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 21:33:32.634101 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.633985 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 21:33:32.634101 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.634008 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dvhrx\"" Jul 08 21:33:32.634101 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.634046 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 21:33:32.638326 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.638213 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d8qct"] Jul 08 21:33:32.638587 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.638545 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-86qgs"] Jul 08 21:33:32.779911 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.779872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-config-volume\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.780096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.779931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfkz7\" (UniqueName: \"kubernetes.io/projected/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-kube-api-access-bfkz7\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.780096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.779973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2922392-a0bd-416b-8189-4c84842e19e0-cert\") pod \"ingress-canary-d8qct\" (UID: \"b2922392-a0bd-416b-8189-4c84842e19e0\") " pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.780096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.779993 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjrgd\" (UniqueName: \"kubernetes.io/projected/b2922392-a0bd-416b-8189-4c84842e19e0-kube-api-access-xjrgd\") pod \"ingress-canary-d8qct\" (UID: \"b2922392-a0bd-416b-8189-4c84842e19e0\") " pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.780257 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.780089 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-tmp-dir\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.780257 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.780118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.881188 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881149 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.881188 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-config-volume\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.881430 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:32.881311 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:33:32.881430 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:32.881386 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls podName:a694888b-76c7-4bc4-93bf-5ad59b2d7bdc nodeName:}" failed. No retries permitted until 2026-07-08 21:33:33.381362239 +0000 UTC m=+36.603372453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls") pod "dns-default-86qgs" (UID: "a694888b-76c7-4bc4-93bf-5ad59b2d7bdc") : secret "dns-default-metrics-tls" not found Jul 08 21:33:32.881627 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfkz7\" (UniqueName: \"kubernetes.io/projected/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-kube-api-access-bfkz7\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.881627 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2922392-a0bd-416b-8189-4c84842e19e0-cert\") pod \"ingress-canary-d8qct\" (UID: \"b2922392-a0bd-416b-8189-4c84842e19e0\") " pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.881627 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjrgd\" (UniqueName: \"kubernetes.io/projected/b2922392-a0bd-416b-8189-4c84842e19e0-kube-api-access-xjrgd\") pod \"ingress-canary-d8qct\" (UID: \"b2922392-a0bd-416b-8189-4c84842e19e0\") " pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.881627 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-tmp-dir\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.881953 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881920 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-tmp-dir\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.881953 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.881949 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-config-volume\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.884411 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.883394 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9tgvh"] Jul 08 21:33:32.887411 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.887296 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2922392-a0bd-416b-8189-4c84842e19e0-cert\") pod \"ingress-canary-d8qct\" (UID: \"b2922392-a0bd-416b-8189-4c84842e19e0\") " pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.887855 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.887840 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:32.893089 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.893065 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 21:33:32.893366 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.893349 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 21:33:32.894191 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.893954 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 21:33:32.894191 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.894079 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 21:33:32.897145 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.896460 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vqcnc\"" Jul 08 21:33:32.898915 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.898883 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfkz7\" (UniqueName: \"kubernetes.io/projected/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-kube-api-access-bfkz7\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:32.900762 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.900737 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjrgd\" (UniqueName: \"kubernetes.io/projected/b2922392-a0bd-416b-8189-4c84842e19e0-kube-api-access-xjrgd\") pod \"ingress-canary-d8qct\" (UID: \"b2922392-a0bd-416b-8189-4c84842e19e0\") " pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.905266 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.905241 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9tgvh"] Jul 08 21:33:32.948763 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.948728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d8qct" Jul 08 21:33:32.982745 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.982701 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:32.982923 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.982800 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:32.982923 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.982831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crjvx\" (UniqueName: \"kubernetes.io/projected/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-api-access-crjvx\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:32.982923 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.982890 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b70cc2a2-566c-4164-82bb-7b264d24c002-crio-socket\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:32.982923 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:32.982915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70cc2a2-566c-4164-82bb-7b264d24c002-data-volume\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.083765 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.083737 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b70cc2a2-566c-4164-82bb-7b264d24c002-crio-socket\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.083963 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.083774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70cc2a2-566c-4164-82bb-7b264d24c002-data-volume\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.083963 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.083923 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.084091 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.084006 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b70cc2a2-566c-4164-82bb-7b264d24c002-crio-socket\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.084091 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.084016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.084091 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.084068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crjvx\" (UniqueName: \"kubernetes.io/projected/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-api-access-crjvx\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.084276 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.084108 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:33:33.084276 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.084173 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70cc2a2-566c-4164-82bb-7b264d24c002-data-volume\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.084276 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.084187 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:33.58416611 +0000 UTC m=+36.806176304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : secret "insights-runtime-extractor-tls" not found Jul 08 21:33:33.084276 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.084247 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:33:33.084455 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.084308 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:33.584289296 +0000 UTC m=+36.806299502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : configmap "kube-rbac-proxy" not found Jul 08 21:33:33.094968 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.094937 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crjvx\" (UniqueName: \"kubernetes.io/projected/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-api-access-crjvx\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.104875 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.104845 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d8qct"] Jul 08 21:33:33.108693 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:33.108668 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2922392_a0bd_416b_8189_4c84842e19e0.slice/crio-8903f6d2f3fc2c4727f22593dd32a99c68d9fd7bf8e97beecae526cab56c9e2f WatchSource:0}: Error finding container 8903f6d2f3fc2c4727f22593dd32a99c68d9fd7bf8e97beecae526cab56c9e2f: Status 404 returned error can't find the container with id 8903f6d2f3fc2c4727f22593dd32a99c68d9fd7bf8e97beecae526cab56c9e2f Jul 08 21:33:33.386842 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.386807 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:33.387543 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.386947 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:33:33.387543 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.387013 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls podName:a694888b-76c7-4bc4-93bf-5ad59b2d7bdc nodeName:}" failed. No retries permitted until 2026-07-08 21:33:34.386996868 +0000 UTC m=+37.609007060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls") pod "dns-default-86qgs" (UID: "a694888b-76c7-4bc4-93bf-5ad59b2d7bdc") : secret "dns-default-metrics-tls" not found Jul 08 21:33:33.553523 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.553483 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d8qct" event={"ID":"b2922392-a0bd-416b-8189-4c84842e19e0","Type":"ContainerStarted","Data":"8903f6d2f3fc2c4727f22593dd32a99c68d9fd7bf8e97beecae526cab56c9e2f"} Jul 08 21:33:33.588573 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.588531 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.588764 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:33.588643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:33.588764 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.588720 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:33:33.588764 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.588733 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:33:33.588898 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.588791 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:34.588775363 +0000 UTC m=+37.810785563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : configmap "kube-rbac-proxy" not found Jul 08 21:33:33.588898 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:33.588804 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:34.588798616 +0000 UTC m=+37.810808794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : secret "insights-runtime-extractor-tls" not found Jul 08 21:33:34.245163 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.244704 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:33:34.245163 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.244729 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:33:34.245163 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.244729 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:33:34.247530 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.247354 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:33:34.247995 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.247974 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:33:34.248119 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.248005 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:33:34.248119 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.248033 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jnwkj\"" Jul 08 21:33:34.248277 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.248148 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:33:34.248277 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.248214 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-khhml\"" Jul 08 21:33:34.396292 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.396250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:34.396863 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:34.396839 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:33:34.396986 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:34.396972 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls podName:a694888b-76c7-4bc4-93bf-5ad59b2d7bdc nodeName:}" failed. No retries permitted until 2026-07-08 21:33:36.396951358 +0000 UTC m=+39.618961537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls") pod "dns-default-86qgs" (UID: "a694888b-76c7-4bc4-93bf-5ad59b2d7bdc") : secret "dns-default-metrics-tls" not found Jul 08 21:33:34.597068 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.596983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:34.597242 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:34.597093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:34.597308 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:34.597242 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:33:34.597308 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:34.597305 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:36.597286761 +0000 UTC m=+39.819296957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : configmap "kube-rbac-proxy" not found Jul 08 21:33:34.597753 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:34.597733 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:33:34.597829 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:34.597779 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:36.597767801 +0000 UTC m=+39.819777979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : secret "insights-runtime-extractor-tls" not found Jul 08 21:33:35.131279 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.131241 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-cj9s6"] Jul 08 21:33:35.134725 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.134694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.136967 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.136946 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 21:33:35.137166 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.137151 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 21:33:35.137673 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.137656 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wfh9n\"" Jul 08 21:33:35.137776 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.137756 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 21:33:35.137832 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.137767 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 21:33:35.142212 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.142192 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-cj9s6"] Jul 08 21:33:35.304033 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.303994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cwbf\" (UniqueName: \"kubernetes.io/projected/6e95653d-5566-48b2-a3f7-17f83e36d932-kube-api-access-4cwbf\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.304222 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.304117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6e95653d-5566-48b2-a3f7-17f83e36d932-signing-key\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.304222 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.304166 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6e95653d-5566-48b2-a3f7-17f83e36d932-signing-cabundle\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.404879 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.404795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6e95653d-5566-48b2-a3f7-17f83e36d932-signing-key\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.404879 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.404829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6e95653d-5566-48b2-a3f7-17f83e36d932-signing-cabundle\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.405442 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.404927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cwbf\" (UniqueName: \"kubernetes.io/projected/6e95653d-5566-48b2-a3f7-17f83e36d932-kube-api-access-4cwbf\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.405771 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.405741 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6e95653d-5566-48b2-a3f7-17f83e36d932-signing-cabundle\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.408186 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.408159 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6e95653d-5566-48b2-a3f7-17f83e36d932-signing-key\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.414883 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.414853 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cwbf\" (UniqueName: \"kubernetes.io/projected/6e95653d-5566-48b2-a3f7-17f83e36d932-kube-api-access-4cwbf\") pod \"service-ca-758f599c5-cj9s6\" (UID: \"6e95653d-5566-48b2-a3f7-17f83e36d932\") " pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.445762 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.445720 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-cj9s6" Jul 08 21:33:35.656384 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:35.656313 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fgr7d_ed16c060-6133-43f3-9eb5-5bbc58e352f0/dns-node-resolver/0.log" Jul 08 21:33:36.413023 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:36.412984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:36.413402 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:36.413152 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:33:36.413402 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:36.413220 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls podName:a694888b-76c7-4bc4-93bf-5ad59b2d7bdc nodeName:}" failed. No retries permitted until 2026-07-08 21:33:40.413203989 +0000 UTC m=+43.635214195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls") pod "dns-default-86qgs" (UID: "a694888b-76c7-4bc4-93bf-5ad59b2d7bdc") : secret "dns-default-metrics-tls" not found Jul 08 21:33:36.463051 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:36.463024 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ddhjc_e6304ba3-31c3-46c9-aa0a-0d9397cbdd27/node-ca/0.log" Jul 08 21:33:36.613995 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:36.613963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:36.614146 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:36.614024 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:36.614146 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:36.614097 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:33:36.614228 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:36.614150 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:33:36.614228 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:36.614182 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:40.61416452 +0000 UTC m=+43.836174698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : configmap "kube-rbac-proxy" not found Jul 08 21:33:36.614228 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:36.614214 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:40.614203978 +0000 UTC m=+43.836214157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : secret "insights-runtime-extractor-tls" not found Jul 08 21:33:36.785091 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:36.784893 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-cj9s6"] Jul 08 21:33:36.790512 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:36.790470 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e95653d_5566_48b2_a3f7_17f83e36d932.slice/crio-1a3ef46cef44477db267f2875cecc6286a712f2849904d32555646b4546ab030 WatchSource:0}: Error finding container 1a3ef46cef44477db267f2875cecc6286a712f2849904d32555646b4546ab030: Status 404 returned error can't find the container with id 1a3ef46cef44477db267f2875cecc6286a712f2849904d32555646b4546ab030 Jul 08 21:33:37.564512 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:37.564478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d8qct" event={"ID":"b2922392-a0bd-416b-8189-4c84842e19e0","Type":"ContainerStarted","Data":"799ec89e64780d4b7ecde55ac0c30beadc8f5e63fb4f23a0f882b3186c038eae"} Jul 08 21:33:37.565604 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:37.565580 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-cj9s6" event={"ID":"6e95653d-5566-48b2-a3f7-17f83e36d932","Type":"ContainerStarted","Data":"1a3ef46cef44477db267f2875cecc6286a712f2849904d32555646b4546ab030"} Jul 08 21:33:37.568407 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:37.568384 2563 generic.go:356] "Generic (PLEG): container finished" podID="6bfb4fef-2218-465f-95a6-8515c5c7c77a" containerID="ebe61705660ba0841bacf9fc5a7928f3c9e95951fc0f13ec28cc57511df46dea" exitCode=0 Jul 08 21:33:37.568515 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:37.568428 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerDied","Data":"ebe61705660ba0841bacf9fc5a7928f3c9e95951fc0f13ec28cc57511df46dea"} Jul 08 21:33:37.582838 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:37.582797 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d8qct" podStartSLOduration=2.051610306 podStartE2EDuration="5.5827822s" totalImagesPullingTime="3.531171894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:37.582361064 +0000 UTC m=+40.804371290" watchObservedRunningTime="2026-07-08 21:33:37.5827822 +0000 UTC m=+40.804792400" Jul 08 21:33:38.578526 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:38.578318 2563 generic.go:356] "Generic (PLEG): container finished" podID="6bfb4fef-2218-465f-95a6-8515c5c7c77a" containerID="561b528638d363caee6c58c4403b6188ff287577fa336a69fb29ac3621da72d1" exitCode=0 Jul 08 21:33:38.579011 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:38.578390 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerDied","Data":"561b528638d363caee6c58c4403b6188ff287577fa336a69fb29ac3621da72d1"} Jul 08 21:33:39.585441 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:39.585403 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" event={"ID":"6bfb4fef-2218-465f-95a6-8515c5c7c77a","Type":"ContainerStarted","Data":"e1be09d59915847c9c4eedeca3417c90b78021b40746a5d949994fb800835cf0"} Jul 08 21:33:40.443567 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.443520 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:40.443791 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:40.443648 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:33:40.443791 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:40.443714 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls podName:a694888b-76c7-4bc4-93bf-5ad59b2d7bdc nodeName:}" failed. No retries permitted until 2026-07-08 21:33:48.443699649 +0000 UTC m=+51.665709827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls") pod "dns-default-86qgs" (UID: "a694888b-76c7-4bc4-93bf-5ad59b2d7bdc") : secret "dns-default-metrics-tls" not found Jul 08 21:33:40.589603 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.589508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-cj9s6" event={"ID":"6e95653d-5566-48b2-a3f7-17f83e36d932","Type":"ContainerStarted","Data":"dada3977f42bbd1147ec01cf04aa9c6c16313b8a1b2ea9d5ad61e7cfdbb22c22"} Jul 08 21:33:40.605182 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.605108 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qh4v8" podStartSLOduration=9.421318727 podStartE2EDuration="42.605088001s" totalImagesPullingTime="33.183769274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:39.61440921 +0000 UTC m=+42.836419414" watchObservedRunningTime="2026-07-08 21:33:40.605088001 +0000 UTC m=+43.827098281" Jul 08 21:33:40.606040 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.605811 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-cj9s6" podStartSLOduration=2.112135737 podStartE2EDuration="5.605798372s" totalImagesPullingTime="3.493662635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:40.604889829 +0000 UTC m=+43.826900031" watchObservedRunningTime="2026-07-08 21:33:40.605798372 +0000 UTC m=+43.827808573" Jul 08 21:33:40.645333 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.645290 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:40.645500 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.645385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:40.645552 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:40.645527 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:33:40.645603 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:33:40.645592 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls podName:b70cc2a2-566c-4164-82bb-7b264d24c002 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:48.645574443 +0000 UTC m=+51.867584635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9tgvh" (UID: "b70cc2a2-566c-4164-82bb-7b264d24c002") : secret "insights-runtime-extractor-tls" not found Jul 08 21:33:40.645844 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:40.645824 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b70cc2a2-566c-4164-82bb-7b264d24c002-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:48.509331 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.509300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:48.513587 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.513540 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a694888b-76c7-4bc4-93bf-5ad59b2d7bdc-metrics-tls\") pod \"dns-default-86qgs\" (UID: \"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc\") " pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:48.542179 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.542144 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:48.707013 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.706981 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-86qgs"] Jul 08 21:33:48.709899 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:48.709877 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda694888b_76c7_4bc4_93bf_5ad59b2d7bdc.slice/crio-80d8f228b237ecccb7cec527cce4e1a2772a35c67200a5611bada366b86d8c07 WatchSource:0}: Error finding container 80d8f228b237ecccb7cec527cce4e1a2772a35c67200a5611bada366b86d8c07: Status 404 returned error can't find the container with id 80d8f228b237ecccb7cec527cce4e1a2772a35c67200a5611bada366b86d8c07 Jul 08 21:33:48.709992 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.709914 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:48.712404 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.712370 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b70cc2a2-566c-4164-82bb-7b264d24c002-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9tgvh\" (UID: \"b70cc2a2-566c-4164-82bb-7b264d24c002\") " pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:48.812624 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.812537 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9tgvh" Jul 08 21:33:48.975923 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:48.975891 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9tgvh"] Jul 08 21:33:48.979084 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:48.979061 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70cc2a2_566c_4164_82bb_7b264d24c002.slice/crio-41d6e7b88379e3b74a0ac36b1facd23b0a83b5386fd38edb1bc16801bff79c21 WatchSource:0}: Error finding container 41d6e7b88379e3b74a0ac36b1facd23b0a83b5386fd38edb1bc16801bff79c21: Status 404 returned error can't find the container with id 41d6e7b88379e3b74a0ac36b1facd23b0a83b5386fd38edb1bc16801bff79c21 Jul 08 21:33:49.617092 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:49.617008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-86qgs" event={"ID":"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc","Type":"ContainerStarted","Data":"80d8f228b237ecccb7cec527cce4e1a2772a35c67200a5611bada366b86d8c07"} Jul 08 21:33:49.618592 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:49.618513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9tgvh" event={"ID":"b70cc2a2-566c-4164-82bb-7b264d24c002","Type":"ContainerStarted","Data":"41d6e7b88379e3b74a0ac36b1facd23b0a83b5386fd38edb1bc16801bff79c21"} Jul 08 21:33:53.631642 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:53.631605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-86qgs" event={"ID":"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc","Type":"ContainerStarted","Data":"57d3c1f792a3037f084fdc759ab508e851d231501cd0956c6459b45679ae9ce8"} Jul 08 21:33:53.633098 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:53.633066 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9tgvh" event={"ID":"b70cc2a2-566c-4164-82bb-7b264d24c002","Type":"ContainerStarted","Data":"1d9bc3cf2a0680e4f1a5c9bd8741cf4a7b022cdd6d166f3f22e072e2cc1a3578"} Jul 08 21:33:54.637857 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:54.637761 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9tgvh" event={"ID":"b70cc2a2-566c-4164-82bb-7b264d24c002","Type":"ContainerStarted","Data":"4a8bf69ae165e0f170552e5bfcf209cdbcc6be71506f1512baf2fa96b87efb12"} Jul 08 21:33:54.639348 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:54.639323 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-86qgs" event={"ID":"a694888b-76c7-4bc4-93bf-5ad59b2d7bdc","Type":"ContainerStarted","Data":"77f940b718a17d8750826c0da4b4e030762730034d3f351b5c2b4e8ca51af047"} Jul 08 21:33:54.639485 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:54.639472 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-86qgs" Jul 08 21:33:54.657037 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:54.656988 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-86qgs" podStartSLOduration=17.913463651 podStartE2EDuration="22.656968874s" totalImagesPullingTime="4.743505223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:54.65621102 +0000 UTC m=+57.878221222" watchObservedRunningTime="2026-07-08 21:33:54.656968874 +0000 UTC m=+57.878979075" Jul 08 21:33:57.652849 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:57.652811 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9tgvh" event={"ID":"b70cc2a2-566c-4164-82bb-7b264d24c002","Type":"ContainerStarted","Data":"c2e4d0833793131de642d04801729cf76e90cb13438b423f162ae58b6da980d5"} Jul 08 21:33:58.557091 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:58.557030 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jvqs" Jul 08 21:33:58.586074 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:58.586015 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9tgvh" podStartSLOduration=18.711580041 podStartE2EDuration="26.585998036s" totalImagesPullingTime="7.874417995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:33:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:57.673320925 +0000 UTC m=+60.895331126" watchObservedRunningTime="2026-07-08 21:33:58.585998036 +0000 UTC m=+61.808008236" Jul 08 21:33:59.027103 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.027058 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2"] Jul 08 21:33:59.046789 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.046750 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp"] Jul 08 21:33:59.046933 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.046887 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.049157 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.049110 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 21:33:59.049157 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.049115 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 21:33:59.049376 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.049110 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 21:33:59.049376 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.049247 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 21:33:59.058823 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.058801 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2"] Jul 08 21:33:59.058935 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.058842 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp"] Jul 08 21:33:59.058935 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.058909 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.060922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.060900 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l8dk9\"" Jul 08 21:33:59.061198 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.061182 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 21:33:59.095806 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.095773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/53b19ef1-b600-441a-80b4-b3b3425d1b0c-klusterlet-config\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.095806 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.095805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8njk\" (UniqueName: \"kubernetes.io/projected/53b19ef1-b600-441a-80b4-b3b3425d1b0c-kube-api-access-n8njk\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.096046 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.095843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwlvn\" (UniqueName: \"kubernetes.io/projected/0e5131f8-22b9-4637-838a-d590b8656299-kube-api-access-mwlvn\") pod \"managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp\" (UID: \"0e5131f8-22b9-4637-838a-d590b8656299\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.096046 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.095865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53b19ef1-b600-441a-80b4-b3b3425d1b0c-tmp\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.096046 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.095884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e5131f8-22b9-4637-838a-d590b8656299-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp\" (UID: \"0e5131f8-22b9-4637-838a-d590b8656299\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.164260 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.164225 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-799698dd98-fc2q4"] Jul 08 21:33:59.180294 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.180265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.182744 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.182718 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 21:33:59.182866 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.182776 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z9lcg\"" Jul 08 21:33:59.182996 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.182980 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 21:33:59.183040 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.183015 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 21:33:59.185310 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.185288 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-799698dd98-fc2q4"] Jul 08 21:33:59.188994 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.188968 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 21:33:59.197037 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197013 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/71f3f18a-f390-42ca-9357-14f8baf01c3d-ca-trust-extracted\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197183 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197044 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxqdr\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-kube-api-access-zxqdr\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197183 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwlvn\" (UniqueName: \"kubernetes.io/projected/0e5131f8-22b9-4637-838a-d590b8656299-kube-api-access-mwlvn\") pod \"managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp\" (UID: \"0e5131f8-22b9-4637-838a-d590b8656299\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.197257 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/71f3f18a-f390-42ca-9357-14f8baf01c3d-image-registry-private-configuration\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197257 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/53b19ef1-b600-441a-80b4-b3b3425d1b0c-klusterlet-config\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.197324 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71f3f18a-f390-42ca-9357-14f8baf01c3d-trusted-ca\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197365 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53b19ef1-b600-441a-80b4-b3b3425d1b0c-tmp\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.197404 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197375 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e5131f8-22b9-4637-838a-d590b8656299-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp\" (UID: \"0e5131f8-22b9-4637-838a-d590b8656299\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.197457 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197403 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-bound-sa-token\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197457 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/71f3f18a-f390-42ca-9357-14f8baf01c3d-installation-pull-secrets\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197551 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197465 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-registry-tls\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197551 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/71f3f18a-f390-42ca-9357-14f8baf01c3d-registry-certificates\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.197653 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8njk\" (UniqueName: \"kubernetes.io/projected/53b19ef1-b600-441a-80b4-b3b3425d1b0c-kube-api-access-n8njk\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.197711 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.197695 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53b19ef1-b600-441a-80b4-b3b3425d1b0c-tmp\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.199754 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.199731 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/53b19ef1-b600-441a-80b4-b3b3425d1b0c-klusterlet-config\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.199917 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.199885 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e5131f8-22b9-4637-838a-d590b8656299-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp\" (UID: \"0e5131f8-22b9-4637-838a-d590b8656299\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.212221 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.212190 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwlvn\" (UniqueName: \"kubernetes.io/projected/0e5131f8-22b9-4637-838a-d590b8656299-kube-api-access-mwlvn\") pod \"managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp\" (UID: \"0e5131f8-22b9-4637-838a-d590b8656299\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.213494 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.213474 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8njk\" (UniqueName: \"kubernetes.io/projected/53b19ef1-b600-441a-80b4-b3b3425d1b0c-kube-api-access-n8njk\") pod \"klusterlet-addon-workmgr-746dcddf58-995g2\" (UID: \"53b19ef1-b600-441a-80b4-b3b3425d1b0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.298113 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298029 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71f3f18a-f390-42ca-9357-14f8baf01c3d-trusted-ca\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298113 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-bound-sa-token\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298113 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/71f3f18a-f390-42ca-9357-14f8baf01c3d-installation-pull-secrets\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298402 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298174 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-registry-tls\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298402 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298214 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/71f3f18a-f390-42ca-9357-14f8baf01c3d-registry-certificates\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298402 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298254 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/71f3f18a-f390-42ca-9357-14f8baf01c3d-ca-trust-extracted\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298402 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxqdr\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-kube-api-access-zxqdr\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298594 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298446 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/71f3f18a-f390-42ca-9357-14f8baf01c3d-image-registry-private-configuration\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.298689 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.298659 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/71f3f18a-f390-42ca-9357-14f8baf01c3d-ca-trust-extracted\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.299184 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.299159 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/71f3f18a-f390-42ca-9357-14f8baf01c3d-registry-certificates\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.299316 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.299302 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71f3f18a-f390-42ca-9357-14f8baf01c3d-trusted-ca\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.300856 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.300833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/71f3f18a-f390-42ca-9357-14f8baf01c3d-image-registry-private-configuration\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.301174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.301156 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-registry-tls\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.301307 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.301288 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/71f3f18a-f390-42ca-9357-14f8baf01c3d-installation-pull-secrets\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.306757 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.306736 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxqdr\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-kube-api-access-zxqdr\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.307001 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.306982 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71f3f18a-f390-42ca-9357-14f8baf01c3d-bound-sa-token\") pod \"image-registry-799698dd98-fc2q4\" (UID: \"71f3f18a-f390-42ca-9357-14f8baf01c3d\") " pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.357915 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.357884 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:33:59.381095 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.381069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" Jul 08 21:33:59.490653 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.490626 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:33:59.533068 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.533033 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2"] Jul 08 21:33:59.535924 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:59.535896 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b19ef1_b600_441a_80b4_b3b3425d1b0c.slice/crio-f2b6509b75da45f65a393483714c585865945b6530556fbb3fecab07875a9ce5 WatchSource:0}: Error finding container f2b6509b75da45f65a393483714c585865945b6530556fbb3fecab07875a9ce5: Status 404 returned error can't find the container with id f2b6509b75da45f65a393483714c585865945b6530556fbb3fecab07875a9ce5 Jul 08 21:33:59.549774 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.549705 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp"] Jul 08 21:33:59.553292 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:59.553265 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e5131f8_22b9_4637_838a_d590b8656299.slice/crio-9ffd4d47417bf62efe2d712a51deb74138721df6007a9873c901f1bbda467b01 WatchSource:0}: Error finding container 9ffd4d47417bf62efe2d712a51deb74138721df6007a9873c901f1bbda467b01: Status 404 returned error can't find the container with id 9ffd4d47417bf62efe2d712a51deb74138721df6007a9873c901f1bbda467b01 Jul 08 21:33:59.658989 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.658967 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-799698dd98-fc2q4"] Jul 08 21:33:59.660946 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.660914 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" event={"ID":"0e5131f8-22b9-4637-838a-d590b8656299","Type":"ContainerStarted","Data":"9ffd4d47417bf62efe2d712a51deb74138721df6007a9873c901f1bbda467b01"} Jul 08 21:33:59.661563 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:33:59.661528 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f3f18a_f390_42ca_9357_14f8baf01c3d.slice/crio-fcd709c6cc570e064713c5104eb7419f57340321a09ad3b34c735f300da7ef0d WatchSource:0}: Error finding container fcd709c6cc570e064713c5104eb7419f57340321a09ad3b34c735f300da7ef0d: Status 404 returned error can't find the container with id fcd709c6cc570e064713c5104eb7419f57340321a09ad3b34c735f300da7ef0d Jul 08 21:33:59.662069 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:33:59.662045 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" event={"ID":"53b19ef1-b600-441a-80b4-b3b3425d1b0c","Type":"ContainerStarted","Data":"f2b6509b75da45f65a393483714c585865945b6530556fbb3fecab07875a9ce5"} Jul 08 21:34:00.667513 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:00.667476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" event={"ID":"71f3f18a-f390-42ca-9357-14f8baf01c3d","Type":"ContainerStarted","Data":"726166a6133afe6b312865930a0bd826f08ff613ced4315740ee740161eff3d1"} Jul 08 21:34:00.667992 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:00.667569 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" event={"ID":"71f3f18a-f390-42ca-9357-14f8baf01c3d","Type":"ContainerStarted","Data":"fcd709c6cc570e064713c5104eb7419f57340321a09ad3b34c735f300da7ef0d"} Jul 08 21:34:00.668248 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:00.668203 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:34:00.691319 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:00.691248 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" podStartSLOduration=1.6912276880000001 podStartE2EDuration="1.691227688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:00.689535527 +0000 UTC m=+63.911545728" watchObservedRunningTime="2026-07-08 21:34:00.691227688 +0000 UTC m=+63.913237890" Jul 08 21:34:02.197698 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.197663 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-kndkl"] Jul 08 21:34:02.200881 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.200857 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:02.205784 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.205756 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 21:34:02.206110 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.206089 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 21:34:02.206248 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.206227 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4pzr6\"" Jul 08 21:34:02.225451 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.225410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvfrb\" (UniqueName: \"kubernetes.io/projected/a29557d8-d381-43f7-8096-c9fc32ce18b3-kube-api-access-jvfrb\") pod \"downloads-685bfb5f64-kndkl\" (UID: \"a29557d8-d381-43f7-8096-c9fc32ce18b3\") " pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:02.266093 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.266056 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-kndkl"] Jul 08 21:34:02.326331 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.326298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvfrb\" (UniqueName: \"kubernetes.io/projected/a29557d8-d381-43f7-8096-c9fc32ce18b3-kube-api-access-jvfrb\") pod \"downloads-685bfb5f64-kndkl\" (UID: \"a29557d8-d381-43f7-8096-c9fc32ce18b3\") " pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:02.340145 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.340102 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvfrb\" (UniqueName: \"kubernetes.io/projected/a29557d8-d381-43f7-8096-c9fc32ce18b3-kube-api-access-jvfrb\") pod \"downloads-685bfb5f64-kndkl\" (UID: \"a29557d8-d381-43f7-8096-c9fc32ce18b3\") " pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:02.512713 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.512623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:02.931833 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.931799 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:34:02.934442 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.934413 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:34:02.945800 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:02.945758 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c-metrics-certs\") pod \"network-metrics-daemon-zqhjt\" (UID: \"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c\") " pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:34:03.032783 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.032736 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:34:03.035377 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.035351 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:34:03.045524 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.045490 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9-original-pull-secret\") pod \"global-pull-secret-syncer-hqqrf\" (UID: \"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9\") " pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:34:03.059735 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.059700 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-khhml\"" Jul 08 21:34:03.068537 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.068496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zqhjt" Jul 08 21:34:03.071419 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.071385 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hqqrf" Jul 08 21:34:03.133784 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.133749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:34:03.136398 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.136372 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:34:03.146638 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.146602 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:34:03.157590 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.157560 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4whd\" (UniqueName: \"kubernetes.io/projected/52e37e95-9c50-4b9b-8fb4-4cf78d479d8d-kube-api-access-c4whd\") pod \"network-check-target-tvbcv\" (UID: \"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d\") " pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:34:03.366962 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.366918 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jnwkj\"" Jul 08 21:34:03.374780 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.374723 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:34:03.594213 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.594183 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hqqrf"] Jul 08 21:34:03.601734 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:03.601709 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1bcf4f_bebf_4346_8da2_5dd2f80d90b9.slice/crio-9fbad23ccd346b7fa5bc3dedf3ea70bf036985c8337ef03ed08ed0be90a8b2ea WatchSource:0}: Error finding container 9fbad23ccd346b7fa5bc3dedf3ea70bf036985c8337ef03ed08ed0be90a8b2ea: Status 404 returned error can't find the container with id 9fbad23ccd346b7fa5bc3dedf3ea70bf036985c8337ef03ed08ed0be90a8b2ea Jul 08 21:34:03.679333 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.679231 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" event={"ID":"53b19ef1-b600-441a-80b4-b3b3425d1b0c","Type":"ContainerStarted","Data":"d9b5595b337740a759b6f6365853f31fb5f0e832273eaafd15eb350eb6ac3d6a"} Jul 08 21:34:03.679507 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.679431 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:34:03.680554 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.680530 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hqqrf" event={"ID":"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9","Type":"ContainerStarted","Data":"9fbad23ccd346b7fa5bc3dedf3ea70bf036985c8337ef03ed08ed0be90a8b2ea"} Jul 08 21:34:03.681189 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.681172 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" Jul 08 21:34:03.694599 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.694547 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-746dcddf58-995g2" podStartSLOduration=0.718482223 podStartE2EDuration="4.694532653s" totalImagesPullingTime="3.97605043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:03.693701793 +0000 UTC m=+66.915711996" watchObservedRunningTime="2026-07-08 21:34:03.694532653 +0000 UTC m=+66.916542854" Jul 08 21:34:03.823446 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.823414 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tvbcv"] Jul 08 21:34:03.826610 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.826561 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zqhjt"] Jul 08 21:34:03.827017 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:03.826992 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e37e95_9c50_4b9b_8fb4_4cf78d479d8d.slice/crio-25aec2be134b23cf95bcda2f425292306457db0fa3f4211857254f498ae28380 WatchSource:0}: Error finding container 25aec2be134b23cf95bcda2f425292306457db0fa3f4211857254f498ae28380: Status 404 returned error can't find the container with id 25aec2be134b23cf95bcda2f425292306457db0fa3f4211857254f498ae28380 Jul 08 21:34:03.828973 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:03.828883 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-kndkl"] Jul 08 21:34:03.831473 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:03.831451 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e42ff9_cf62_4ff2_96e5_2dabbdac2e7c.slice/crio-68734076fd4c8b9f99127c4184ef437e9b054f0a0387e66b023e0f16a85a26fc WatchSource:0}: Error finding container 68734076fd4c8b9f99127c4184ef437e9b054f0a0387e66b023e0f16a85a26fc: Status 404 returned error can't find the container with id 68734076fd4c8b9f99127c4184ef437e9b054f0a0387e66b023e0f16a85a26fc Jul 08 21:34:03.832907 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:03.832888 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29557d8_d381_43f7_8096_c9fc32ce18b3.slice/crio-8cc98cdf0558fb83fc84914c437ba99e9566657429a7b5d291d44e728935a6d2 WatchSource:0}: Error finding container 8cc98cdf0558fb83fc84914c437ba99e9566657429a7b5d291d44e728935a6d2: Status 404 returned error can't find the container with id 8cc98cdf0558fb83fc84914c437ba99e9566657429a7b5d291d44e728935a6d2 Jul 08 21:34:04.647660 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:04.647062 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-86qgs" Jul 08 21:34:04.688100 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:04.687968 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tvbcv" event={"ID":"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d","Type":"ContainerStarted","Data":"25aec2be134b23cf95bcda2f425292306457db0fa3f4211857254f498ae28380"} Jul 08 21:34:04.693430 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:04.693364 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-kndkl" event={"ID":"a29557d8-d381-43f7-8096-c9fc32ce18b3","Type":"ContainerStarted","Data":"8cc98cdf0558fb83fc84914c437ba99e9566657429a7b5d291d44e728935a6d2"} Jul 08 21:34:04.695875 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:04.695845 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zqhjt" event={"ID":"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c","Type":"ContainerStarted","Data":"68734076fd4c8b9f99127c4184ef437e9b054f0a0387e66b023e0f16a85a26fc"} Jul 08 21:34:05.701986 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:05.701886 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zqhjt" event={"ID":"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c","Type":"ContainerStarted","Data":"38991981d47d29a309b2df3e7db0f2c93244afa6b2fae59036fade9a7f24c255"} Jul 08 21:34:05.701986 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:05.701971 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zqhjt" event={"ID":"95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c","Type":"ContainerStarted","Data":"d447aa10f4deeecccdd6085fe3817fecafc1829955aec4db83e4cf0e02a66f1b"} Jul 08 21:34:05.720114 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:05.718878 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zqhjt" podStartSLOduration=66.363243559 podStartE2EDuration="1m7.718854907s" totalImagesPullingTime="1.355611348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:05.718199499 +0000 UTC m=+68.940209698" watchObservedRunningTime="2026-07-08 21:34:05.718854907 +0000 UTC m=+68.940865109" Jul 08 21:34:10.725681 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.725634 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hqqrf" event={"ID":"1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9","Type":"ContainerStarted","Data":"4c7981bb79977499d0ba0985c25bd0b0e85e4406040462ed04dda5aa10ebb13d"} Jul 08 21:34:10.727440 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.727413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tvbcv" event={"ID":"52e37e95-9c50-4b9b-8fb4-4cf78d479d8d","Type":"ContainerStarted","Data":"adf36b726cc9d3036bc7d7841d811a840870248509c309918a6cd87d643da566"} Jul 08 21:34:10.727590 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.727517 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:34:10.729120 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.729090 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" event={"ID":"0e5131f8-22b9-4637-838a-d590b8656299","Type":"ContainerStarted","Data":"3a659077f323b26251d4fbe7997c0021c450e66dc3d84204729a9d60b6ee6273"} Jul 08 21:34:10.741571 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.741520 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hqqrf" podStartSLOduration=66.479950641 podStartE2EDuration="1m12.741504844s" totalImagesPullingTime="6.261554203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:10.740318607 +0000 UTC m=+73.962328810" watchObservedRunningTime="2026-07-08 21:34:10.741504844 +0000 UTC m=+73.963515024" Jul 08 21:34:10.755280 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.755230 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f6f7d499d-rlpvp" podStartSLOduration=1.3805676789999999 podStartE2EDuration="11.755212613s" totalImagesPullingTime="10.374644934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:10.754177345 +0000 UTC m=+73.976187536" watchObservedRunningTime="2026-07-08 21:34:10.755212613 +0000 UTC m=+73.977222814" Jul 08 21:34:10.768649 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.768585 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tvbcv" podStartSLOduration=66.732473941 podStartE2EDuration="1m12.768562691s" totalImagesPullingTime="6.03608875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:10.768075259 +0000 UTC m=+73.990085460" watchObservedRunningTime="2026-07-08 21:34:10.768562691 +0000 UTC m=+73.990572893" Jul 08 21:34:10.918179 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.918142 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66488c4955-57qhh"] Jul 08 21:34:10.921398 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.921376 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:10.923809 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.923770 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 21:34:10.923941 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.923924 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 21:34:10.924033 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.924016 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 21:34:10.924358 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.924234 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qv5s8\"" Jul 08 21:34:10.924358 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.924264 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 21:34:10.924358 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.924292 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 21:34:10.931244 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.931219 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66488c4955-57qhh"] Jul 08 21:34:10.996830 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.996730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-oauth-serving-cert\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:10.996830 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.996784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpvtd\" (UniqueName: \"kubernetes.io/projected/6953c3e2-9b25-47ec-9634-92f1845b8a72-kube-api-access-bpvtd\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:10.997062 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.996869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-config\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:10.997062 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.996912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-service-ca\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:10.997062 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.996957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-serving-cert\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:10.997062 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:10.997002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-oauth-config\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098240 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098200 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-oauth-serving-cert\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098240 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098240 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpvtd\" (UniqueName: \"kubernetes.io/projected/6953c3e2-9b25-47ec-9634-92f1845b8a72-kube-api-access-bpvtd\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098493 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-config\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098493 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-service-ca\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098493 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-serving-cert\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098493 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-oauth-config\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.098963 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.098923 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-oauth-serving-cert\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.099150 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.099111 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-service-ca\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.099323 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.099300 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-config\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.100873 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.100854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-oauth-config\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.101189 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.101174 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-serving-cert\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.106601 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.106577 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpvtd\" (UniqueName: \"kubernetes.io/projected/6953c3e2-9b25-47ec-9634-92f1845b8a72-kube-api-access-bpvtd\") pod \"console-66488c4955-57qhh\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.232991 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.232944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:11.383291 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.383252 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66488c4955-57qhh"] Jul 08 21:34:11.387502 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:11.387474 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6953c3e2_9b25_47ec_9634_92f1845b8a72.slice/crio-abdc4b5bdcc4a76b641bd75fae78624fa617230ac470b33bfd34921c72a02449 WatchSource:0}: Error finding container abdc4b5bdcc4a76b641bd75fae78624fa617230ac470b33bfd34921c72a02449: Status 404 returned error can't find the container with id abdc4b5bdcc4a76b641bd75fae78624fa617230ac470b33bfd34921c72a02449 Jul 08 21:34:11.734574 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:11.734524 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66488c4955-57qhh" event={"ID":"6953c3e2-9b25-47ec-9634-92f1845b8a72","Type":"ContainerStarted","Data":"abdc4b5bdcc4a76b641bd75fae78624fa617230ac470b33bfd34921c72a02449"} Jul 08 21:34:12.046235 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.046068 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5vpzn"] Jul 08 21:34:12.050461 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.050430 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.053399 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.053165 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 21:34:12.055722 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.055273 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 21:34:12.056180 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.056013 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lvlkr\"" Jul 08 21:34:12.058658 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.057436 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 21:34:12.058658 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.057669 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 21:34:12.058658 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.057829 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 21:34:12.058658 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.058013 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 21:34:12.058905 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.058817 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7"] Jul 08 21:34:12.065348 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.064766 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.069488 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.067978 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 21:34:12.069488 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.068228 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 21:34:12.069488 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.068400 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-s2vxl\"" Jul 08 21:34:12.069488 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.068633 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 21:34:12.080472 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.080369 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7"] Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.106991 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/989a4096-a50c-4828-a7c3-9e3845df3a4b-metrics-client-ca\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef134186-603f-4d09-9547-7edc81b22799-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107100 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-accelerators-collector-config\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107154 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9twhn\" (UniqueName: \"kubernetes.io/projected/ef134186-603f-4d09-9547-7edc81b22799-kube-api-access-9twhn\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-wtmp\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107216 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzpxw\" (UniqueName: \"kubernetes.io/projected/989a4096-a50c-4828-a7c3-9e3845df3a4b-kube-api-access-qzpxw\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107275 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-root\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-tls\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107352 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-sys\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107402 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-textfile\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107432 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.107644 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.107454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef134186-603f-4d09-9547-7edc81b22799-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.208830 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208794 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-accelerators-collector-config\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209005 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9twhn\" (UniqueName: \"kubernetes.io/projected/ef134186-603f-4d09-9547-7edc81b22799-kube-api-access-9twhn\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.209005 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208879 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-wtmp\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209005 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzpxw\" (UniqueName: \"kubernetes.io/projected/989a4096-a50c-4828-a7c3-9e3845df3a4b-kube-api-access-qzpxw\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209005 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-root\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209005 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-tls\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209005 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.208989 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-sys\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-textfile\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef134186-603f-4d09-9547-7edc81b22799-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/989a4096-a50c-4828-a7c3-9e3845df3a4b-metrics-client-ca\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef134186-603f-4d09-9547-7edc81b22799-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.209329 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.209244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.210038 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.210013 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.210539 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.210516 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-accelerators-collector-config\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.210983 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.210942 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-wtmp\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.211219 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.211200 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-root\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.211293 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:34:12.211285 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 21:34:12.211344 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:34:12.211337 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-tls podName:989a4096-a50c-4828-a7c3-9e3845df3a4b nodeName:}" failed. No retries permitted until 2026-07-08 21:34:12.711320658 +0000 UTC m=+75.933330843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-tls") pod "node-exporter-5vpzn" (UID: "989a4096-a50c-4828-a7c3-9e3845df3a4b") : secret "node-exporter-tls" not found Jul 08 21:34:12.212924 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.212874 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/989a4096-a50c-4828-a7c3-9e3845df3a4b-metrics-client-ca\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.213197 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.213177 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ef134186-603f-4d09-9547-7edc81b22799-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.213720 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.213697 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef134186-603f-4d09-9547-7edc81b22799-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.213803 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.213772 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/989a4096-a50c-4828-a7c3-9e3845df3a4b-sys\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.214017 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.213997 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-textfile\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.215646 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.215589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.216888 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.216765 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef134186-603f-4d09-9547-7edc81b22799-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.222330 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.222282 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzpxw\" (UniqueName: \"kubernetes.io/projected/989a4096-a50c-4828-a7c3-9e3845df3a4b-kube-api-access-qzpxw\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.226714 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.226661 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.226845 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.226709 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9twhn\" (UniqueName: \"kubernetes.io/projected/ef134186-603f-4d09-9547-7edc81b22799-kube-api-access-9twhn\") pod \"kube-state-metrics-7c9dbb8b7d-jltb7\" (UID: \"ef134186-603f-4d09-9547-7edc81b22799\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.387547 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.387095 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" Jul 08 21:34:12.603164 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.603117 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7"] Jul 08 21:34:12.605914 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:12.605870 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef134186_603f_4d09_9547_7edc81b22799.slice/crio-2aa6f3fb9d9e9a68d9967edc828d9e58fd4bf43354de75199a87670a03accd4b WatchSource:0}: Error finding container 2aa6f3fb9d9e9a68d9967edc828d9e58fd4bf43354de75199a87670a03accd4b: Status 404 returned error can't find the container with id 2aa6f3fb9d9e9a68d9967edc828d9e58fd4bf43354de75199a87670a03accd4b Jul 08 21:34:12.712818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.712721 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-tls\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.715569 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.715546 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/989a4096-a50c-4828-a7c3-9e3845df3a4b-node-exporter-tls\") pod \"node-exporter-5vpzn\" (UID: \"989a4096-a50c-4828-a7c3-9e3845df3a4b\") " pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.741791 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.741753 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" event={"ID":"ef134186-603f-4d09-9547-7edc81b22799","Type":"ContainerStarted","Data":"2aa6f3fb9d9e9a68d9967edc828d9e58fd4bf43354de75199a87670a03accd4b"} Jul 08 21:34:12.969751 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:12.969663 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5vpzn" Jul 08 21:34:12.988756 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:12.988716 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989a4096_a50c_4828_a7c3_9e3845df3a4b.slice/crio-f846d9e4f54258868396c741088b0b0ea9360de4eaca3acf8d6582796bf8d0ac WatchSource:0}: Error finding container f846d9e4f54258868396c741088b0b0ea9360de4eaca3acf8d6582796bf8d0ac: Status 404 returned error can't find the container with id f846d9e4f54258868396c741088b0b0ea9360de4eaca3acf8d6582796bf8d0ac Jul 08 21:34:13.752238 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:13.752189 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vpzn" event={"ID":"989a4096-a50c-4828-a7c3-9e3845df3a4b","Type":"ContainerStarted","Data":"f846d9e4f54258868396c741088b0b0ea9360de4eaca3acf8d6582796bf8d0ac"} Jul 08 21:34:16.767874 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.767828 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" event={"ID":"ef134186-603f-4d09-9547-7edc81b22799","Type":"ContainerStarted","Data":"f390480bb53ea4596a3005eefe9c0b9463970fca29b9ba76bac76de5cd350114"} Jul 08 21:34:16.769943 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.769887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66488c4955-57qhh" event={"ID":"6953c3e2-9b25-47ec-9634-92f1845b8a72","Type":"ContainerStarted","Data":"1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2"} Jul 08 21:34:16.773732 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.773700 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vpzn" event={"ID":"989a4096-a50c-4828-a7c3-9e3845df3a4b","Type":"ContainerStarted","Data":"8e5b20bc63f5b4a4af2ba20faa00c0c0a323884fe80a9a68e53cd5d6e094db8b"} Jul 08 21:34:16.790062 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.789819 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66488c4955-57qhh" podStartSLOduration=2.379177825 podStartE2EDuration="6.789799864s" totalImagesPullingTime="4.410622039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:16.788606869 +0000 UTC m=+80.010617076" watchObservedRunningTime="2026-07-08 21:34:16.789799864 +0000 UTC m=+80.011810069" Jul 08 21:34:16.828503 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.828464 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq"] Jul 08 21:34:16.832032 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.832007 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:16.834281 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.834252 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 21:34:16.834419 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.834281 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pzwnb\"" Jul 08 21:34:16.842634 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.842607 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq"] Jul 08 21:34:16.953751 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.953714 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:16.953908 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:16.953777 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjnfc\" (UniqueName: \"kubernetes.io/projected/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-kube-api-access-sjnfc\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:17.054749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:17.054667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:17.054749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:17.054739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjnfc\" (UniqueName: \"kubernetes.io/projected/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-kube-api-access-sjnfc\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:17.054983 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:34:17.054846 2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 21:34:17.054983 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:34:17.054928 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-monitoring-plugin-cert podName:fa65a4d2-0fcb-424b-810d-6f4fe3c88302 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:17.554907241 +0000 UTC m=+80.776917426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-6l9lq" (UID: "fa65a4d2-0fcb-424b-810d-6f4fe3c88302") : secret "monitoring-plugin-cert" not found Jul 08 21:34:17.064400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:17.064369 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjnfc\" (UniqueName: \"kubernetes.io/projected/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-kube-api-access-sjnfc\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:17.559655 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:17.559499 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:17.562516 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:17.562489 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa65a4d2-0fcb-424b-810d-6f4fe3c88302-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6l9lq\" (UID: \"fa65a4d2-0fcb-424b-810d-6f4fe3c88302\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:17.743618 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:17.743588 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:19.308634 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.307518 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fc5448b6d-dwwsx"] Jul 08 21:34:19.474360 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.474316 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc5448b6d-dwwsx"] Jul 08 21:34:19.474547 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.474392 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.482217 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.482191 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 21:34:19.578271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-trusted-ca-bundle\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.578271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578228 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmpg\" (UniqueName: \"kubernetes.io/projected/15e9f777-f268-43f6-961d-587e80df6cc4-kube-api-access-wnmpg\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.578491 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-oauth-serving-cert\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.578491 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578304 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-serving-cert\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.578491 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-service-ca\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.578491 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578378 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-oauth-config\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.578491 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.578430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-console-config\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679066 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-service-ca\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679274 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-oauth-config\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679274 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-console-config\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679274 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-trusted-ca-bundle\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679274 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmpg\" (UniqueName: \"kubernetes.io/projected/15e9f777-f268-43f6-961d-587e80df6cc4-kube-api-access-wnmpg\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679274 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-oauth-serving-cert\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679519 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679277 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-serving-cert\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.679888 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679858 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-service-ca\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.680048 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.679925 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-console-config\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.680499 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.680454 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-oauth-serving-cert\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.680640 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.680552 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-trusted-ca-bundle\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.682060 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.682040 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-oauth-config\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.682576 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.682558 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-serving-cert\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.687702 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.687681 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmpg\" (UniqueName: \"kubernetes.io/projected/15e9f777-f268-43f6-961d-587e80df6cc4-kube-api-access-wnmpg\") pod \"console-6fc5448b6d-dwwsx\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:19.786186 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:19.786153 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:21.234078 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:21.234034 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:21.234669 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:21.234113 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:21.239690 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:21.239661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:21.797980 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:21.797946 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:22.679876 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:22.679845 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-799698dd98-fc2q4" Jul 08 21:34:23.303094 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.303062 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc5448b6d-dwwsx"] Jul 08 21:34:23.307486 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:23.307446 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e9f777_f268_43f6_961d_587e80df6cc4.slice/crio-1e7306c035f40a376570b1d8cc8ca63f241c3f7a5bda701e83c269e1e87302b9 WatchSource:0}: Error finding container 1e7306c035f40a376570b1d8cc8ca63f241c3f7a5bda701e83c269e1e87302b9: Status 404 returned error can't find the container with id 1e7306c035f40a376570b1d8cc8ca63f241c3f7a5bda701e83c269e1e87302b9 Jul 08 21:34:23.324979 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.324953 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq"] Jul 08 21:34:23.336263 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:34:23.336233 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa65a4d2_0fcb_424b_810d_6f4fe3c88302.slice/crio-956cd5cbf0a5a36c191ab3ec5629381508e9f4b57df5c8176baa938a14729426 WatchSource:0}: Error finding container 956cd5cbf0a5a36c191ab3ec5629381508e9f4b57df5c8176baa938a14729426: Status 404 returned error can't find the container with id 956cd5cbf0a5a36c191ab3ec5629381508e9f4b57df5c8176baa938a14729426 Jul 08 21:34:23.801171 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.801118 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" event={"ID":"ef134186-603f-4d09-9547-7edc81b22799","Type":"ContainerStarted","Data":"ea356e3c123012bd3eaff568f58e69bd6ef623c9db4f262689800657af639233"} Jul 08 21:34:23.801726 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.801196 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" event={"ID":"ef134186-603f-4d09-9547-7edc81b22799","Type":"ContainerStarted","Data":"8b9c4974f7541a1f915178720362a5b533d2de40b878d60032a3b277940c8aa9"} Jul 08 21:34:23.802301 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.802279 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" event={"ID":"fa65a4d2-0fcb-424b-810d-6f4fe3c88302","Type":"ContainerStarted","Data":"956cd5cbf0a5a36c191ab3ec5629381508e9f4b57df5c8176baa938a14729426"} Jul 08 21:34:23.803623 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.803602 2563 generic.go:356] "Generic (PLEG): container finished" podID="989a4096-a50c-4828-a7c3-9e3845df3a4b" containerID="8e5b20bc63f5b4a4af2ba20faa00c0c0a323884fe80a9a68e53cd5d6e094db8b" exitCode=0 Jul 08 21:34:23.803718 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.803675 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vpzn" event={"ID":"989a4096-a50c-4828-a7c3-9e3845df3a4b","Type":"ContainerDied","Data":"8e5b20bc63f5b4a4af2ba20faa00c0c0a323884fe80a9a68e53cd5d6e094db8b"} Jul 08 21:34:23.805209 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.805187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5448b6d-dwwsx" event={"ID":"15e9f777-f268-43f6-961d-587e80df6cc4","Type":"ContainerStarted","Data":"6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86"} Jul 08 21:34:23.805295 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.805220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5448b6d-dwwsx" event={"ID":"15e9f777-f268-43f6-961d-587e80df6cc4","Type":"ContainerStarted","Data":"1e7306c035f40a376570b1d8cc8ca63f241c3f7a5bda701e83c269e1e87302b9"} Jul 08 21:34:23.820677 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.820625 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-jltb7" podStartSLOduration=8.635951596 podStartE2EDuration="11.820609597s" totalImagesPullingTime="3.184658001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:23.818748538 +0000 UTC m=+87.040758739" watchObservedRunningTime="2026-07-08 21:34:23.820609597 +0000 UTC m=+87.042619798" Jul 08 21:34:23.856577 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:23.856524 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fc5448b6d-dwwsx" podStartSLOduration=4.85650823 podStartE2EDuration="4.85650823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:23.855355212 +0000 UTC m=+87.077365421" watchObservedRunningTime="2026-07-08 21:34:23.85650823 +0000 UTC m=+87.078518410" Jul 08 21:34:24.813954 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.813907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-kndkl" event={"ID":"a29557d8-d381-43f7-8096-c9fc32ce18b3","Type":"ContainerStarted","Data":"5cd633dd72228ad35d519267053a55ba439acc2ed1edeb3dd9c8c01e5c5ccbea"} Jul 08 21:34:24.814450 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.813987 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:24.816930 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.816904 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vpzn" event={"ID":"989a4096-a50c-4828-a7c3-9e3845df3a4b","Type":"ContainerStarted","Data":"a7cb80c8a6d036960b3c057cb3ae9499da8a33c26dc671f32893f09f251d2b34"} Jul 08 21:34:24.817073 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.816964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vpzn" event={"ID":"989a4096-a50c-4828-a7c3-9e3845df3a4b","Type":"ContainerStarted","Data":"acc9a5697b968bae6daa3e77ddd6d2dff5efa2b25fa21607fcb697b19be3d0fb"} Jul 08 21:34:24.833319 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.833246 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-kndkl" podStartSLOduration=2.7389393589999997 podStartE2EDuration="22.833225574s" totalImagesPullingTime="20.094286215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:24.830897801 +0000 UTC m=+88.052907989" watchObservedRunningTime="2026-07-08 21:34:24.833225574 +0000 UTC m=+88.055235776" Jul 08 21:34:24.836454 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.836421 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-kndkl" Jul 08 21:34:24.870415 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:24.870348 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5vpzn" podStartSLOduration=9.974085205 podStartE2EDuration="12.870326661s" totalImagesPullingTime="2.802432417s" totalInitContainerRuntime="93.809039ms" isStatefulPod=true podCreationTimestamp="2026-07-08 21:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:24.869615871 +0000 UTC m=+88.091626073" watchObservedRunningTime="2026-07-08 21:34:24.870326661 +0000 UTC m=+88.092336863" Jul 08 21:34:25.823852 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:25.823773 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" event={"ID":"fa65a4d2-0fcb-424b-810d-6f4fe3c88302","Type":"ContainerStarted","Data":"7153be8dd09e783fb83ce9755f08666599a3ccb7cbacd8555acd788c8497bf19"} Jul 08 21:34:25.824436 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:25.824409 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:25.830344 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:25.830321 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" Jul 08 21:34:25.844714 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:25.844650 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6l9lq" podStartSLOduration=7.935157472 podStartE2EDuration="9.844630054s" totalImagesPullingTime="1.909472582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:25.843684481 +0000 UTC m=+89.065694684" watchObservedRunningTime="2026-07-08 21:34:25.844630054 +0000 UTC m=+89.066640256" Jul 08 21:34:29.786616 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:29.786580 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:29.787383 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:29.786843 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:29.792581 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:29.792554 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:29.846092 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:29.846060 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:34:29.895060 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:29.895022 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66488c4955-57qhh"] Jul 08 21:34:41.737531 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:41.737501 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tvbcv" Jul 08 21:34:54.923468 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:54.923402 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66488c4955-57qhh" podUID="6953c3e2-9b25-47ec-9634-92f1845b8a72" containerName="console" containerID="cri-o://1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2" gracePeriod=15 Jul 08 21:34:55.204145 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.204059 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:55.277689 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.277660 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-service-ca\") pod \"6953c3e2-9b25-47ec-9634-92f1845b8a72\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " Jul 08 21:34:55.277878 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.277705 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-oauth-config\") pod \"6953c3e2-9b25-47ec-9634-92f1845b8a72\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " Jul 08 21:34:55.277878 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.277739 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-config\") pod \"6953c3e2-9b25-47ec-9634-92f1845b8a72\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " Jul 08 21:34:55.277878 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.277769 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-oauth-serving-cert\") pod \"6953c3e2-9b25-47ec-9634-92f1845b8a72\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " Jul 08 21:34:55.277878 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.277791 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-serving-cert\") pod \"6953c3e2-9b25-47ec-9634-92f1845b8a72\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " Jul 08 21:34:55.277878 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.277827 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpvtd\" (UniqueName: \"kubernetes.io/projected/6953c3e2-9b25-47ec-9634-92f1845b8a72-kube-api-access-bpvtd\") pod \"6953c3e2-9b25-47ec-9634-92f1845b8a72\" (UID: \"6953c3e2-9b25-47ec-9634-92f1845b8a72\") " Jul 08 21:34:55.278257 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.278221 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-service-ca" (OuterVolumeSpecName: "service-ca") pod "6953c3e2-9b25-47ec-9634-92f1845b8a72" (UID: "6953c3e2-9b25-47ec-9634-92f1845b8a72"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:34:55.278399 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.278261 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-config" (OuterVolumeSpecName: "console-config") pod "6953c3e2-9b25-47ec-9634-92f1845b8a72" (UID: "6953c3e2-9b25-47ec-9634-92f1845b8a72"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:34:55.278399 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.278291 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6953c3e2-9b25-47ec-9634-92f1845b8a72" (UID: "6953c3e2-9b25-47ec-9634-92f1845b8a72"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:34:55.280262 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.280221 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6953c3e2-9b25-47ec-9634-92f1845b8a72-kube-api-access-bpvtd" (OuterVolumeSpecName: "kube-api-access-bpvtd") pod "6953c3e2-9b25-47ec-9634-92f1845b8a72" (UID: "6953c3e2-9b25-47ec-9634-92f1845b8a72"). InnerVolumeSpecName "kube-api-access-bpvtd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:34:55.280341 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.280296 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6953c3e2-9b25-47ec-9634-92f1845b8a72" (UID: "6953c3e2-9b25-47ec-9634-92f1845b8a72"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:34:55.280341 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.280314 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6953c3e2-9b25-47ec-9634-92f1845b8a72" (UID: "6953c3e2-9b25-47ec-9634-92f1845b8a72"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:34:55.379174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.379107 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:34:55.379174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.379167 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-oauth-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:34:55.379174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.379177 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:34:55.379174 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.379186 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpvtd\" (UniqueName: \"kubernetes.io/projected/6953c3e2-9b25-47ec-9634-92f1845b8a72-kube-api-access-bpvtd\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:34:55.379444 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.379195 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6953c3e2-9b25-47ec-9634-92f1845b8a72-service-ca\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:34:55.379444 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.379204 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6953c3e2-9b25-47ec-9634-92f1845b8a72-console-oauth-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:34:55.932271 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.932234 2563 generic.go:356] "Generic (PLEG): container finished" podID="6953c3e2-9b25-47ec-9634-92f1845b8a72" containerID="1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2" exitCode=0 Jul 08 21:34:55.932662 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.932297 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66488c4955-57qhh" Jul 08 21:34:55.932662 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.932316 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66488c4955-57qhh" event={"ID":"6953c3e2-9b25-47ec-9634-92f1845b8a72","Type":"ContainerDied","Data":"1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2"} Jul 08 21:34:55.932662 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.932389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66488c4955-57qhh" event={"ID":"6953c3e2-9b25-47ec-9634-92f1845b8a72","Type":"ContainerDied","Data":"abdc4b5bdcc4a76b641bd75fae78624fa617230ac470b33bfd34921c72a02449"} Jul 08 21:34:55.932662 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.932409 2563 scope.go:117] "RemoveContainer" containerID="1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2" Jul 08 21:34:55.940973 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.940956 2563 scope.go:117] "RemoveContainer" containerID="1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2" Jul 08 21:34:55.941261 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:34:55.941239 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2\": container with ID starting with 1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2 not found: ID does not exist" containerID="1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2" Jul 08 21:34:55.941316 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.941270 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2"} err="failed to get container status \"1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2\": rpc error: code = NotFound desc = could not find container \"1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2\": container with ID starting with 1d990d05bf9118598fe953d9afe7e38b822eb3fe23fa373bc22066285198bfe2 not found: ID does not exist" Jul 08 21:34:55.952704 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.952675 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66488c4955-57qhh"] Jul 08 21:34:55.958139 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:55.958108 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66488c4955-57qhh"] Jul 08 21:34:57.249400 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:34:57.249363 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6953c3e2-9b25-47ec-9634-92f1845b8a72" path="/var/lib/kubelet/pods/6953c3e2-9b25-47ec-9634-92f1845b8a72/volumes" Jul 08 21:35:40.397744 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.397708 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8dd9475c-s8bxk"] Jul 08 21:35:40.398197 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.397958 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6953c3e2-9b25-47ec-9634-92f1845b8a72" containerName="console" Jul 08 21:35:40.398197 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.397968 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6953c3e2-9b25-47ec-9634-92f1845b8a72" containerName="console" Jul 08 21:35:40.398197 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.398014 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6953c3e2-9b25-47ec-9634-92f1845b8a72" containerName="console" Jul 08 21:35:40.401076 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.401058 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.411586 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.411559 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dd9475c-s8bxk"] Jul 08 21:35:40.535792 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-config\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.535792 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prh5z\" (UniqueName: \"kubernetes.io/projected/d5338f7b-a54a-4042-acbb-b1e102dea9b6-kube-api-access-prh5z\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.536027 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-oauth-config\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.536027 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-service-ca\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.536027 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-trusted-ca-bundle\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.536027 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-oauth-serving-cert\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.536027 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.535989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-serving-cert\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.636688 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-service-ca\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.636688 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-trusted-ca-bundle\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.636922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-oauth-serving-cert\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.636922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-serving-cert\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.636922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636776 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-config\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.636922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636793 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prh5z\" (UniqueName: \"kubernetes.io/projected/d5338f7b-a54a-4042-acbb-b1e102dea9b6-kube-api-access-prh5z\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.637178 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.636974 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-oauth-config\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.637487 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.637463 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-service-ca\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.637856 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.637763 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-oauth-serving-cert\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.637943 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.637850 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-trusted-ca-bundle\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.637943 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.637932 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-config\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.639466 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.639444 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-serving-cert\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.639597 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.639575 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-oauth-config\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.644715 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.644693 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prh5z\" (UniqueName: \"kubernetes.io/projected/d5338f7b-a54a-4042-acbb-b1e102dea9b6-kube-api-access-prh5z\") pod \"console-8dd9475c-s8bxk\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.710288 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.710210 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:40.840083 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:40.840054 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dd9475c-s8bxk"] Jul 08 21:35:40.843369 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:35:40.843337 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5338f7b_a54a_4042_acbb_b1e102dea9b6.slice/crio-2f12a52c5239c1541e8e5a717a59124e238d93ba37bbb514c25b7cabee948f86 WatchSource:0}: Error finding container 2f12a52c5239c1541e8e5a717a59124e238d93ba37bbb514c25b7cabee948f86: Status 404 returned error can't find the container with id 2f12a52c5239c1541e8e5a717a59124e238d93ba37bbb514c25b7cabee948f86 Jul 08 21:35:41.081387 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:41.081297 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dd9475c-s8bxk" event={"ID":"d5338f7b-a54a-4042-acbb-b1e102dea9b6","Type":"ContainerStarted","Data":"78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff"} Jul 08 21:35:41.081387 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:41.081350 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dd9475c-s8bxk" event={"ID":"d5338f7b-a54a-4042-acbb-b1e102dea9b6","Type":"ContainerStarted","Data":"2f12a52c5239c1541e8e5a717a59124e238d93ba37bbb514c25b7cabee948f86"} Jul 08 21:35:41.101116 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:41.101061 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8dd9475c-s8bxk" podStartSLOduration=1.101043822 podStartE2EDuration="1.101043822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:35:41.100218266 +0000 UTC m=+164.322228467" watchObservedRunningTime="2026-07-08 21:35:41.101043822 +0000 UTC m=+164.323054024" Jul 08 21:35:50.710764 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:50.710672 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:50.711282 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:50.711008 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:50.715549 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:50.715529 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:51.119914 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:51.119882 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:35:51.157707 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:35:51.157672 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc5448b6d-dwwsx"] Jul 08 21:36:16.184517 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.184452 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6fc5448b6d-dwwsx" podUID="15e9f777-f268-43f6-961d-587e80df6cc4" containerName="console" containerID="cri-o://6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86" gracePeriod=15 Jul 08 21:36:16.434896 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.434830 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:36:16.498351 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498316 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-oauth-config\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498351 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498355 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-service-ca\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498591 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498381 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-oauth-serving-cert\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498591 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498506 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-console-config\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498591 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498545 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-serving-cert\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498591 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498574 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-trusted-ca-bundle\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498846 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498598 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnmpg\" (UniqueName: \"kubernetes.io/projected/15e9f777-f268-43f6-961d-587e80df6cc4-kube-api-access-wnmpg\") pod \"15e9f777-f268-43f6-961d-587e80df6cc4\" (UID: \"15e9f777-f268-43f6-961d-587e80df6cc4\") " Jul 08 21:36:16.498900 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498876 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:36:16.498943 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498903 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-service-ca" (OuterVolumeSpecName: "service-ca") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:36:16.498996 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.498971 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-console-config" (OuterVolumeSpecName: "console-config") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:36:16.499145 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.499103 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:36:16.500706 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.500685 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:36:16.500785 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.500706 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:36:16.500821 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.500788 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15e9f777-f268-43f6-961d-587e80df6cc4-kube-api-access-wnmpg" (OuterVolumeSpecName: "kube-api-access-wnmpg") pod "15e9f777-f268-43f6-961d-587e80df6cc4" (UID: "15e9f777-f268-43f6-961d-587e80df6cc4"). InnerVolumeSpecName "kube-api-access-wnmpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:36:16.599965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.599926 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-oauth-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:16.599965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.599959 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-service-ca\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:16.599965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.599969 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-oauth-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:16.600243 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.599980 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-console-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:16.600243 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.599988 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15e9f777-f268-43f6-961d-587e80df6cc4-console-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:16.600243 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.599995 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15e9f777-f268-43f6-961d-587e80df6cc4-trusted-ca-bundle\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:16.600243 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:16.600005 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnmpg\" (UniqueName: \"kubernetes.io/projected/15e9f777-f268-43f6-961d-587e80df6cc4-kube-api-access-wnmpg\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:36:17.207624 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.207589 2563 generic.go:356] "Generic (PLEG): container finished" podID="15e9f777-f268-43f6-961d-587e80df6cc4" containerID="6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86" exitCode=0 Jul 08 21:36:17.208019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.207644 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc5448b6d-dwwsx" Jul 08 21:36:17.208019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.207682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5448b6d-dwwsx" event={"ID":"15e9f777-f268-43f6-961d-587e80df6cc4","Type":"ContainerDied","Data":"6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86"} Jul 08 21:36:17.208019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.207771 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc5448b6d-dwwsx" event={"ID":"15e9f777-f268-43f6-961d-587e80df6cc4","Type":"ContainerDied","Data":"1e7306c035f40a376570b1d8cc8ca63f241c3f7a5bda701e83c269e1e87302b9"} Jul 08 21:36:17.208019 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.207796 2563 scope.go:117] "RemoveContainer" containerID="6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86" Jul 08 21:36:17.215988 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.215970 2563 scope.go:117] "RemoveContainer" containerID="6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86" Jul 08 21:36:17.216272 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:36:17.216253 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86\": container with ID starting with 6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86 not found: ID does not exist" containerID="6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86" Jul 08 21:36:17.216317 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.216281 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86"} err="failed to get container status \"6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86\": rpc error: code = NotFound desc = could not find container \"6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86\": container with ID starting with 6eed76c53867ea61f3f733f11eb86a1c557869b58b9de7bc05bdd37548a3fa86 not found: ID does not exist" Jul 08 21:36:17.229903 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.229875 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc5448b6d-dwwsx"] Jul 08 21:36:17.235703 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.235678 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fc5448b6d-dwwsx"] Jul 08 21:36:17.248885 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:36:17.248861 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15e9f777-f268-43f6-961d-587e80df6cc4" path="/var/lib/kubelet/pods/15e9f777-f268-43f6-961d-587e80df6cc4/volumes" Jul 08 21:37:02.404902 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.404861 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d5ffdb76-n2qdp"] Jul 08 21:37:02.405447 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.405297 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15e9f777-f268-43f6-961d-587e80df6cc4" containerName="console" Jul 08 21:37:02.405447 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.405315 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="15e9f777-f268-43f6-961d-587e80df6cc4" containerName="console" Jul 08 21:37:02.405447 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.405378 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="15e9f777-f268-43f6-961d-587e80df6cc4" containerName="console" Jul 08 21:37:02.409450 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.409428 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.420450 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.420420 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d5ffdb76-n2qdp"] Jul 08 21:37:02.546454 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-serving-cert\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.546454 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-trusted-ca-bundle\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.546703 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-config\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.546703 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546611 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-oauth-serving-cert\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.546703 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-oauth-config\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.546703 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546658 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8m7q\" (UniqueName: \"kubernetes.io/projected/6f55df5c-5107-4b8c-922e-ac5db54c9908-kube-api-access-d8m7q\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.546703 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.546684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-service-ca\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.647918 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.647873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-config\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.647918 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.647920 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-oauth-serving-cert\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648327 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.647939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-oauth-config\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648327 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8m7q\" (UniqueName: \"kubernetes.io/projected/6f55df5c-5107-4b8c-922e-ac5db54c9908-kube-api-access-d8m7q\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648327 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-service-ca\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648327 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-serving-cert\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648327 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648196 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-trusted-ca-bundle\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648826 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648800 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-oauth-serving-cert\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648800 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-service-ca\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.648977 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.648926 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-config\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.649256 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.649233 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-trusted-ca-bundle\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.650562 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.650540 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-oauth-config\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.650780 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.650762 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-serving-cert\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.658116 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.658065 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8m7q\" (UniqueName: \"kubernetes.io/projected/6f55df5c-5107-4b8c-922e-ac5db54c9908-kube-api-access-d8m7q\") pod \"console-6d5ffdb76-n2qdp\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.719069 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.719024 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:02.853773 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:02.853734 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d5ffdb76-n2qdp"] Jul 08 21:37:02.857984 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:37:02.857959 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f55df5c_5107_4b8c_922e_ac5db54c9908.slice/crio-417d9a68797e1e1cd0085abb100e93fc8d14f058e530296fc5ed719500990216 WatchSource:0}: Error finding container 417d9a68797e1e1cd0085abb100e93fc8d14f058e530296fc5ed719500990216: Status 404 returned error can't find the container with id 417d9a68797e1e1cd0085abb100e93fc8d14f058e530296fc5ed719500990216 Jul 08 21:37:03.364608 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:03.364572 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ffdb76-n2qdp" event={"ID":"6f55df5c-5107-4b8c-922e-ac5db54c9908","Type":"ContainerStarted","Data":"50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161"} Jul 08 21:37:03.364799 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:03.364622 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ffdb76-n2qdp" event={"ID":"6f55df5c-5107-4b8c-922e-ac5db54c9908","Type":"ContainerStarted","Data":"417d9a68797e1e1cd0085abb100e93fc8d14f058e530296fc5ed719500990216"} Jul 08 21:37:03.393202 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:03.393152 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d5ffdb76-n2qdp" podStartSLOduration=1.393101063 podStartE2EDuration="1.393101063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:37:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:37:03.391166306 +0000 UTC m=+246.613176504" watchObservedRunningTime="2026-07-08 21:37:03.393101063 +0000 UTC m=+246.615111268" Jul 08 21:37:12.719528 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:12.719483 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:12.720022 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:12.719557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:12.724224 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:12.724202 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:13.404958 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:13.404925 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:37:13.449639 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:13.449607 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dd9475c-s8bxk"] Jul 08 21:37:38.473265 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.473228 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8dd9475c-s8bxk" podUID="d5338f7b-a54a-4042-acbb-b1e102dea9b6" containerName="console" containerID="cri-o://78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff" gracePeriod=15 Jul 08 21:37:38.712681 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.712659 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:37:38.717552 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717524 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-oauth-config\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.717691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717561 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-service-ca\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.717691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717588 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-serving-cert\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.717691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717608 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-oauth-serving-cert\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.717691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717649 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prh5z\" (UniqueName: \"kubernetes.io/projected/d5338f7b-a54a-4042-acbb-b1e102dea9b6-kube-api-access-prh5z\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.717691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717667 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-trusted-ca-bundle\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.717922 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717718 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-config\") pod \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\" (UID: \"d5338f7b-a54a-4042-acbb-b1e102dea9b6\") " Jul 08 21:37:38.718175 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.717971 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-service-ca" (OuterVolumeSpecName: "service-ca") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:37:38.718317 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.718233 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-config" (OuterVolumeSpecName: "console-config") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:37:38.718317 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.718225 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:37:38.718939 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.718910 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:37:38.720038 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.720017 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:37:38.720197 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.720173 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:37:38.720260 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.720198 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5338f7b-a54a-4042-acbb-b1e102dea9b6-kube-api-access-prh5z" (OuterVolumeSpecName: "kube-api-access-prh5z") pod "d5338f7b-a54a-4042-acbb-b1e102dea9b6" (UID: "d5338f7b-a54a-4042-acbb-b1e102dea9b6"). InnerVolumeSpecName "kube-api-access-prh5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:37:38.819065 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819038 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:38.819065 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819062 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-oauth-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:38.819065 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819072 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prh5z\" (UniqueName: \"kubernetes.io/projected/d5338f7b-a54a-4042-acbb-b1e102dea9b6-kube-api-access-prh5z\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:38.819311 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819082 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-trusted-ca-bundle\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:38.819311 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819091 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:38.819311 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819099 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5338f7b-a54a-4042-acbb-b1e102dea9b6-console-oauth-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:38.819311 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:38.819107 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5338f7b-a54a-4042-acbb-b1e102dea9b6-service-ca\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:37:39.490613 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.490580 2563 generic.go:356] "Generic (PLEG): container finished" podID="d5338f7b-a54a-4042-acbb-b1e102dea9b6" containerID="78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff" exitCode=0 Jul 08 21:37:39.490998 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.490629 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dd9475c-s8bxk" event={"ID":"d5338f7b-a54a-4042-acbb-b1e102dea9b6","Type":"ContainerDied","Data":"78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff"} Jul 08 21:37:39.490998 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.490644 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dd9475c-s8bxk" Jul 08 21:37:39.490998 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.490688 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dd9475c-s8bxk" event={"ID":"d5338f7b-a54a-4042-acbb-b1e102dea9b6","Type":"ContainerDied","Data":"2f12a52c5239c1541e8e5a717a59124e238d93ba37bbb514c25b7cabee948f86"} Jul 08 21:37:39.490998 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.490714 2563 scope.go:117] "RemoveContainer" containerID="78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff" Jul 08 21:37:39.498930 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.498911 2563 scope.go:117] "RemoveContainer" containerID="78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff" Jul 08 21:37:39.499212 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:37:39.499194 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff\": container with ID starting with 78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff not found: ID does not exist" containerID="78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff" Jul 08 21:37:39.499276 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.499220 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff"} err="failed to get container status \"78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff\": rpc error: code = NotFound desc = could not find container \"78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff\": container with ID starting with 78d6ac891bfe1919be98202b77e6eee97d79b2b5f19b37bf06cbdb74ba8ad0ff not found: ID does not exist" Jul 08 21:37:39.512355 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.512300 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dd9475c-s8bxk"] Jul 08 21:37:39.514688 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:39.514666 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8dd9475c-s8bxk"] Jul 08 21:37:41.249561 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:41.249519 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5338f7b-a54a-4042-acbb-b1e102dea9b6" path="/var/lib/kubelet/pods/d5338f7b-a54a-4042-acbb-b1e102dea9b6/volumes" Jul 08 21:37:57.108403 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:37:57.108378 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 21:39:58.939555 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.939515 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tc4sw"] Jul 08 21:39:58.940081 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.939915 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5338f7b-a54a-4042-acbb-b1e102dea9b6" containerName="console" Jul 08 21:39:58.940081 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.939932 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5338f7b-a54a-4042-acbb-b1e102dea9b6" containerName="console" Jul 08 21:39:58.940081 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.940003 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5338f7b-a54a-4042-acbb-b1e102dea9b6" containerName="console" Jul 08 21:39:58.942914 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.942893 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:58.945894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.945869 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 21:39:58.946075 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.945869 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5f4pk\"" Jul 08 21:39:58.946215 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.946200 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 21:39:58.959012 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:58.958981 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tc4sw"] Jul 08 21:39:59.015453 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.015423 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d66a5f86-37e9-448a-b114-dd3d1eb15b94-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tc4sw\" (UID: \"d66a5f86-37e9-448a-b114-dd3d1eb15b94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.015621 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.015458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnbxt\" (UniqueName: \"kubernetes.io/projected/d66a5f86-37e9-448a-b114-dd3d1eb15b94-kube-api-access-qnbxt\") pod \"cert-manager-webhook-6486ffd997-tc4sw\" (UID: \"d66a5f86-37e9-448a-b114-dd3d1eb15b94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.116013 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.115979 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d66a5f86-37e9-448a-b114-dd3d1eb15b94-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tc4sw\" (UID: \"d66a5f86-37e9-448a-b114-dd3d1eb15b94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.116187 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.116017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnbxt\" (UniqueName: \"kubernetes.io/projected/d66a5f86-37e9-448a-b114-dd3d1eb15b94-kube-api-access-qnbxt\") pod \"cert-manager-webhook-6486ffd997-tc4sw\" (UID: \"d66a5f86-37e9-448a-b114-dd3d1eb15b94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.124170 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.124144 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d66a5f86-37e9-448a-b114-dd3d1eb15b94-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tc4sw\" (UID: \"d66a5f86-37e9-448a-b114-dd3d1eb15b94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.124341 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.124322 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnbxt\" (UniqueName: \"kubernetes.io/projected/d66a5f86-37e9-448a-b114-dd3d1eb15b94-kube-api-access-qnbxt\") pod \"cert-manager-webhook-6486ffd997-tc4sw\" (UID: \"d66a5f86-37e9-448a-b114-dd3d1eb15b94\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.251784 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.251699 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:39:59.381932 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.381897 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tc4sw"] Jul 08 21:39:59.384766 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:39:59.384732 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd66a5f86_37e9_448a_b114_dd3d1eb15b94.slice/crio-99f0e3aa1584920a71bdccddeffc7685b8edcaa6b8116e355cacafce9217f155 WatchSource:0}: Error finding container 99f0e3aa1584920a71bdccddeffc7685b8edcaa6b8116e355cacafce9217f155: Status 404 returned error can't find the container with id 99f0e3aa1584920a71bdccddeffc7685b8edcaa6b8116e355cacafce9217f155 Jul 08 21:39:59.386796 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.386779 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:39:59.969346 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:39:59.969313 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" event={"ID":"d66a5f86-37e9-448a-b114-dd3d1eb15b94","Type":"ContainerStarted","Data":"99f0e3aa1584920a71bdccddeffc7685b8edcaa6b8116e355cacafce9217f155"} Jul 08 21:40:00.579567 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.579530 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5blgz"] Jul 08 21:40:00.582948 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.582921 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.585544 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.585329 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-54fx8\"" Jul 08 21:40:00.596316 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.596244 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5blgz"] Jul 08 21:40:00.628693 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.628614 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbbh9\" (UniqueName: \"kubernetes.io/projected/50f90a93-2660-4e1c-883c-5d9504bcd50e-kube-api-access-vbbh9\") pod \"cert-manager-cainjector-559cdd8c87-5blgz\" (UID: \"50f90a93-2660-4e1c-883c-5d9504bcd50e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.628879 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.628765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f90a93-2660-4e1c-883c-5d9504bcd50e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5blgz\" (UID: \"50f90a93-2660-4e1c-883c-5d9504bcd50e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.730171 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.730084 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f90a93-2660-4e1c-883c-5d9504bcd50e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5blgz\" (UID: \"50f90a93-2660-4e1c-883c-5d9504bcd50e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.730388 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.730184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbbh9\" (UniqueName: \"kubernetes.io/projected/50f90a93-2660-4e1c-883c-5d9504bcd50e-kube-api-access-vbbh9\") pod \"cert-manager-cainjector-559cdd8c87-5blgz\" (UID: \"50f90a93-2660-4e1c-883c-5d9504bcd50e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.739688 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.739635 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f90a93-2660-4e1c-883c-5d9504bcd50e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5blgz\" (UID: \"50f90a93-2660-4e1c-883c-5d9504bcd50e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.739946 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.739894 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbbh9\" (UniqueName: \"kubernetes.io/projected/50f90a93-2660-4e1c-883c-5d9504bcd50e-kube-api-access-vbbh9\") pod \"cert-manager-cainjector-559cdd8c87-5blgz\" (UID: \"50f90a93-2660-4e1c-883c-5d9504bcd50e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:00.897592 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:00.897555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" Jul 08 21:40:01.061527 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:01.061487 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5blgz"] Jul 08 21:40:01.065512 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:40:01.065477 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50f90a93_2660_4e1c_883c_5d9504bcd50e.slice/crio-3ec161f490526f86dd67b97ccca6ffeb6d6a13f0c395bf7cd37e50e61a583834 WatchSource:0}: Error finding container 3ec161f490526f86dd67b97ccca6ffeb6d6a13f0c395bf7cd37e50e61a583834: Status 404 returned error can't find the container with id 3ec161f490526f86dd67b97ccca6ffeb6d6a13f0c395bf7cd37e50e61a583834 Jul 08 21:40:01.980892 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:01.980847 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" event={"ID":"50f90a93-2660-4e1c-883c-5d9504bcd50e","Type":"ContainerStarted","Data":"3ec161f490526f86dd67b97ccca6ffeb6d6a13f0c395bf7cd37e50e61a583834"} Jul 08 21:40:02.987167 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:02.987109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" event={"ID":"d66a5f86-37e9-448a-b114-dd3d1eb15b94","Type":"ContainerStarted","Data":"eca0045ffe03684fa307abf40234e8cd83c4e353cf03f8ccadc29b4e94ba40e2"} Jul 08 21:40:02.987631 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:02.987215 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:40:02.988864 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:02.988835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" event={"ID":"50f90a93-2660-4e1c-883c-5d9504bcd50e","Type":"ContainerStarted","Data":"cdf246cb9c27fd9dce5affef7e9716038661f674dea5166ac98399699e755e6e"} Jul 08 21:40:03.010833 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:03.010780 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" podStartSLOduration=1.509609145 podStartE2EDuration="5.010763897s" totalImagesPullingTime="3.501154752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:40:03.009445537 +0000 UTC m=+426.231455738" watchObservedRunningTime="2026-07-08 21:40:03.010763897 +0000 UTC m=+426.232774098" Jul 08 21:40:03.026453 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:03.026401 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5blgz" podStartSLOduration=1.206374637 podStartE2EDuration="3.02638013s" totalImagesPullingTime="1.820005493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:40:03.024852655 +0000 UTC m=+426.246862859" watchObservedRunningTime="2026-07-08 21:40:03.02638013 +0000 UTC m=+426.248390336" Jul 08 21:40:08.995431 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:08.995394 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tc4sw" Jul 08 21:40:35.393876 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.393840 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7"] Jul 08 21:40:35.398605 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.398584 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.401017 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.400994 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 21:40:35.401239 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.401057 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 21:40:35.401239 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.401153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 21:40:35.401239 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.401168 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 21:40:35.401458 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.401276 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jnwhn\"" Jul 08 21:40:35.414342 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.414320 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7"] Jul 08 21:40:35.497412 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.497368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.497613 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.497459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-tmp\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.497613 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.497549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlrsb\" (UniqueName: \"kubernetes.io/projected/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-kube-api-access-tlrsb\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.497613 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.497588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.497767 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.497638 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-manifests\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.598840 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.598802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.598840 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.598848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-tmp\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.599100 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.598890 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlrsb\" (UniqueName: \"kubernetes.io/projected/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-kube-api-access-tlrsb\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.599100 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.598909 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.599100 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.598935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-manifests\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.599425 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.599403 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-manifests\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.601375 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.601350 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-tmp\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.601614 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.601591 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.601675 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.601660 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.612377 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.612343 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlrsb\" (UniqueName: \"kubernetes.io/projected/b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d-kube-api-access-tlrsb\") pod \"opendatahub-operator-controller-manager-697f9f6747-b56m7\" (UID: \"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d\") " pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.709312 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.709226 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:35.843930 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:35.843896 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7"] Jul 08 21:40:35.847825 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:40:35.847795 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22585eb_ee29_40ce_a8fb_9c2c4cd33c5d.slice/crio-75913e3143c9cb5f63344b8c303c447627f1a4bb8e45e7c65363bff1b4f59bf3 WatchSource:0}: Error finding container 75913e3143c9cb5f63344b8c303c447627f1a4bb8e45e7c65363bff1b4f59bf3: Status 404 returned error can't find the container with id 75913e3143c9cb5f63344b8c303c447627f1a4bb8e45e7c65363bff1b4f59bf3 Jul 08 21:40:36.107764 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:36.107735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" event={"ID":"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d","Type":"ContainerStarted","Data":"75913e3143c9cb5f63344b8c303c447627f1a4bb8e45e7c65363bff1b4f59bf3"} Jul 08 21:40:39.121728 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:39.121636 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" event={"ID":"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d","Type":"ContainerStarted","Data":"ffb5ffb0ee107ba0def7a32491d6f4b9f2e2f18b2cad14d575ff40208c3e920d"} Jul 08 21:40:40.126621 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:40.126581 2563 generic.go:356] "Generic (PLEG): container finished" podID="b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d" containerID="ffb5ffb0ee107ba0def7a32491d6f4b9f2e2f18b2cad14d575ff40208c3e920d" exitCode=0 Jul 08 21:40:40.127047 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:40.126625 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" event={"ID":"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d","Type":"ContainerDied","Data":"ffb5ffb0ee107ba0def7a32491d6f4b9f2e2f18b2cad14d575ff40208c3e920d"} Jul 08 21:40:41.133030 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:41.132992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" event={"ID":"b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d","Type":"ContainerStarted","Data":"84e4021b16f0000e53d905c034f2110ce2792629aaf0bf6bb18a59f3c3226cdf"} Jul 08 21:40:41.133565 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:41.133158 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:41.158046 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:41.157940 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" podStartSLOduration=1.651549391 podStartE2EDuration="6.15792009s" totalImagesPullingTime="3.41732042s" totalInitContainerRuntime="1.089050279s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:40:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:40:41.156503742 +0000 UTC m=+464.378513945" watchObservedRunningTime="2026-07-08 21:40:41.15792009 +0000 UTC m=+464.379930300" Jul 08 21:40:48.253659 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.253624 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f58fdc776-q76d9"] Jul 08 21:40:48.265147 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.265092 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.266778 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.266754 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f58fdc776-q76d9"] Jul 08 21:40:48.409558 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409522 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-oauth-serving-cert\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.409558 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27cdaf17-d31c-4760-9328-20996b5f04bf-console-oauth-config\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.409818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfmbv\" (UniqueName: \"kubernetes.io/projected/27cdaf17-d31c-4760-9328-20996b5f04bf-kube-api-access-bfmbv\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.409818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-trusted-ca-bundle\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.409818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409721 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-service-ca\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.409818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-console-config\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.409818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.409812 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27cdaf17-d31c-4760-9328-20996b5f04bf-console-serving-cert\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511108 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-service-ca\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511108 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-console-config\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511108 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27cdaf17-d31c-4760-9328-20996b5f04bf-console-serving-cert\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511108 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-oauth-serving-cert\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511437 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27cdaf17-d31c-4760-9328-20996b5f04bf-console-oauth-config\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511437 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfmbv\" (UniqueName: \"kubernetes.io/projected/27cdaf17-d31c-4760-9328-20996b5f04bf-kube-api-access-bfmbv\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511437 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-trusted-ca-bundle\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511952 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511921 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-oauth-serving-cert\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.511952 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.511921 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-console-config\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.512182 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.512162 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-trusted-ca-bundle\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.512245 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.512184 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27cdaf17-d31c-4760-9328-20996b5f04bf-service-ca\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.513742 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.513712 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27cdaf17-d31c-4760-9328-20996b5f04bf-console-serving-cert\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.513851 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.513818 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27cdaf17-d31c-4760-9328-20996b5f04bf-console-oauth-config\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.518890 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.518869 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfmbv\" (UniqueName: \"kubernetes.io/projected/27cdaf17-d31c-4760-9328-20996b5f04bf-kube-api-access-bfmbv\") pod \"console-7f58fdc776-q76d9\" (UID: \"27cdaf17-d31c-4760-9328-20996b5f04bf\") " pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.575214 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.575184 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:48.713780 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:48.713713 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f58fdc776-q76d9"] Jul 08 21:40:48.716543 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:40:48.716514 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27cdaf17_d31c_4760_9328_20996b5f04bf.slice/crio-6f37c8a91819f09b157f39bf70fd9e8ba220c209931d31f02ca49a13910610e4 WatchSource:0}: Error finding container 6f37c8a91819f09b157f39bf70fd9e8ba220c209931d31f02ca49a13910610e4: Status 404 returned error can't find the container with id 6f37c8a91819f09b157f39bf70fd9e8ba220c209931d31f02ca49a13910610e4 Jul 08 21:40:49.164187 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:49.164156 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f58fdc776-q76d9" event={"ID":"27cdaf17-d31c-4760-9328-20996b5f04bf","Type":"ContainerStarted","Data":"4e33dd372840897eb9dbbeb8bc9a9f49ca8a51b9e57bcc7fc655c49b7a920e3c"} Jul 08 21:40:49.164366 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:49.164204 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f58fdc776-q76d9" event={"ID":"27cdaf17-d31c-4760-9328-20996b5f04bf","Type":"ContainerStarted","Data":"6f37c8a91819f09b157f39bf70fd9e8ba220c209931d31f02ca49a13910610e4"} Jul 08 21:40:49.210652 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:49.210590 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f58fdc776-q76d9" podStartSLOduration=1.210569117 podStartE2EDuration="1.210569117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:40:49.209693944 +0000 UTC m=+472.431704145" watchObservedRunningTime="2026-07-08 21:40:49.210569117 +0000 UTC m=+472.432579320" Jul 08 21:40:52.140120 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:52.140089 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-697f9f6747-b56m7" Jul 08 21:40:55.271538 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.271503 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr"] Jul 08 21:40:55.279236 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.279207 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.284500 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.284475 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zlmq7\"" Jul 08 21:40:55.286054 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.285900 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 21:40:55.286331 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.286314 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 21:40:55.286435 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.286406 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 21:40:55.286713 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.286700 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 21:40:55.294663 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.294638 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr"] Jul 08 21:40:55.468040 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.468004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35861ca-8595-414c-bf05-cbe70c2b6586-tls-certs\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.468248 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.468048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f8q4\" (UniqueName: \"kubernetes.io/projected/f35861ca-8595-414c-bf05-cbe70c2b6586-kube-api-access-4f8q4\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.468248 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.468149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35861ca-8595-414c-bf05-cbe70c2b6586-tmp\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.569318 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.569226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35861ca-8595-414c-bf05-cbe70c2b6586-tmp\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.569318 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.569283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35861ca-8595-414c-bf05-cbe70c2b6586-tls-certs\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.569318 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.569313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f8q4\" (UniqueName: \"kubernetes.io/projected/f35861ca-8595-414c-bf05-cbe70c2b6586-kube-api-access-4f8q4\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.571971 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.571943 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35861ca-8595-414c-bf05-cbe70c2b6586-tmp\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.572171 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.572153 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f35861ca-8595-414c-bf05-cbe70c2b6586-tls-certs\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.600208 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.600171 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f8q4\" (UniqueName: \"kubernetes.io/projected/f35861ca-8595-414c-bf05-cbe70c2b6586-kube-api-access-4f8q4\") pod \"kube-auth-proxy-9dfdc66fb-msdqr\" (UID: \"f35861ca-8595-414c-bf05-cbe70c2b6586\") " pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:55.889162 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:55.889110 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" Jul 08 21:40:56.044055 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.044028 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr"] Jul 08 21:40:56.046451 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:40:56.046419 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35861ca_8595_414c_bf05_cbe70c2b6586.slice/crio-58ccbbd641b14d8de05558c53402cad2e2739248d6045fa6c3a1771146aa08d5 WatchSource:0}: Error finding container 58ccbbd641b14d8de05558c53402cad2e2739248d6045fa6c3a1771146aa08d5: Status 404 returned error can't find the container with id 58ccbbd641b14d8de05558c53402cad2e2739248d6045fa6c3a1771146aa08d5 Jul 08 21:40:56.193760 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.193677 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" event={"ID":"f35861ca-8595-414c-bf05-cbe70c2b6586","Type":"ContainerStarted","Data":"58ccbbd641b14d8de05558c53402cad2e2739248d6045fa6c3a1771146aa08d5"} Jul 08 21:40:56.695201 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.695164 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7wdjv"] Jul 08 21:40:56.699282 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.698899 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:56.701357 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.701336 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 21:40:56.717798 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.717749 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7wdjv"] Jul 08 21:40:56.780098 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.779951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6bgz\" (UniqueName: \"kubernetes.io/projected/f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249-kube-api-access-j6bgz\") pod \"model-serving-api-54d89bd5fc-7wdjv\" (UID: \"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249\") " pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:56.780098 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.780012 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249-tls-certs\") pod \"model-serving-api-54d89bd5fc-7wdjv\" (UID: \"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249\") " pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:56.881487 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.881454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6bgz\" (UniqueName: \"kubernetes.io/projected/f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249-kube-api-access-j6bgz\") pod \"model-serving-api-54d89bd5fc-7wdjv\" (UID: \"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249\") " pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:56.881669 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.881512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249-tls-certs\") pod \"model-serving-api-54d89bd5fc-7wdjv\" (UID: \"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249\") " pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:56.884570 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.884542 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249-tls-certs\") pod \"model-serving-api-54d89bd5fc-7wdjv\" (UID: \"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249\") " pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:56.892901 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:56.892869 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6bgz\" (UniqueName: \"kubernetes.io/projected/f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249-kube-api-access-j6bgz\") pod \"model-serving-api-54d89bd5fc-7wdjv\" (UID: \"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249\") " pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:57.012029 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:57.011944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:40:57.158560 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:57.158523 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7wdjv"] Jul 08 21:40:57.162138 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:40:57.162093 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4dd7d70_9c8d_40ca_b6df_5f8ab8a28249.slice/crio-2915a774d1348f0af159dbe12b92ef24fa0758e3a03426bd4305a02812a4405c WatchSource:0}: Error finding container 2915a774d1348f0af159dbe12b92ef24fa0758e3a03426bd4305a02812a4405c: Status 404 returned error can't find the container with id 2915a774d1348f0af159dbe12b92ef24fa0758e3a03426bd4305a02812a4405c Jul 08 21:40:57.198102 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:57.198071 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" event={"ID":"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249","Type":"ContainerStarted","Data":"2915a774d1348f0af159dbe12b92ef24fa0758e3a03426bd4305a02812a4405c"} Jul 08 21:40:58.576007 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:58.575916 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:58.576517 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:58.576074 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:58.581995 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:58.581965 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:59.191096 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:59.191078 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 21:40:59.214969 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:59.214947 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f58fdc776-q76d9" Jul 08 21:40:59.263383 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:40:59.263360 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d5ffdb76-n2qdp"] Jul 08 21:41:00.214749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:00.214707 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" event={"ID":"f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249","Type":"ContainerStarted","Data":"43c5fcfcf51b139b9ac5f5d2c01b0ad4947ce2c2187de15d9a82ea6912100994"} Jul 08 21:41:00.215270 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:00.214807 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:41:00.216513 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:00.216487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" event={"ID":"f35861ca-8595-414c-bf05-cbe70c2b6586","Type":"ContainerStarted","Data":"e971df3667a8307d6d01528d605e979e044ac09d0c94dfd2f6ae574d76a3b7f9"} Jul 08 21:41:00.230766 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:00.230716 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" podStartSLOduration=2.206254388 podStartE2EDuration="4.23070005s" totalImagesPullingTime="2.024445662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:41:00.229896331 +0000 UTC m=+483.451906532" watchObservedRunningTime="2026-07-08 21:41:00.23070005 +0000 UTC m=+483.452710251" Jul 08 21:41:00.245581 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:00.245535 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9dfdc66fb-msdqr" podStartSLOduration=2.104999524 podStartE2EDuration="5.24551905s" totalImagesPullingTime="3.140519526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:41:00.244090272 +0000 UTC m=+483.466100471" watchObservedRunningTime="2026-07-08 21:41:00.24551905 +0000 UTC m=+483.467529251" Jul 08 21:41:02.470115 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.470073 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm"] Jul 08 21:41:02.473913 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.473894 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.476746 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.476721 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mnp8t\"" Jul 08 21:41:02.477067 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.477052 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 21:41:02.490234 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.490206 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm"] Jul 08 21:41:02.528474 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528647 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528647 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528647 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trftr\" (UniqueName: \"kubernetes.io/projected/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-kube-api-access-trftr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528647 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528885 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528674 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528885 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528885 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.528885 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.528795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.629944 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.629887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.629944 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.629946 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.629992 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trftr\" (UniqueName: \"kubernetes.io/projected/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-kube-api-access-trftr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630063 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630254 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630229 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630642 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630392 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630642 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630531 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630642 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630568 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630794 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630648 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.630794 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.630776 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.632751 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.632729 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.632989 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.632971 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.640247 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.640221 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trftr\" (UniqueName: \"kubernetes.io/projected/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-kube-api-access-trftr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.640869 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.640850 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa754c7e-32e9-4b85-a87c-31f2aa76b87c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llcm\" (UID: \"aa754c7e-32e9-4b85-a87c-31f2aa76b87c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.784742 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.784655 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:02.971371 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:02.971347 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm"] Jul 08 21:41:02.973275 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:41:02.973232 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa754c7e_32e9_4b85_a87c_31f2aa76b87c.slice/crio-0d83bc2cbfb3ca348db4d358c6fb3eb0e145329ff295e2d3bc817592f8b627a9 WatchSource:0}: Error finding container 0d83bc2cbfb3ca348db4d358c6fb3eb0e145329ff295e2d3bc817592f8b627a9: Status 404 returned error can't find the container with id 0d83bc2cbfb3ca348db4d358c6fb3eb0e145329ff295e2d3bc817592f8b627a9 Jul 08 21:41:03.210488 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.210455 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5947469944-fx6bz"] Jul 08 21:41:03.214840 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.214816 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.217054 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.217026 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:41:03.217216 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.217092 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 21:41:03.217216 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.217109 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bp25t\"" Jul 08 21:41:03.217216 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.217094 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 21:41:03.217216 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.217164 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 21:41:03.217216 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.217181 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 21:41:03.228155 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.225352 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5947469944-fx6bz"] Jul 08 21:41:03.233474 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.233441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqqbz\" (UniqueName: \"kubernetes.io/projected/d426944a-6e3b-4177-a1b5-64b6626c28ef-kube-api-access-dqqbz\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.233749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.233727 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d426944a-6e3b-4177-a1b5-64b6626c28ef-metrics-cert\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.233911 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.233895 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d426944a-6e3b-4177-a1b5-64b6626c28ef-manager-config\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.234063 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.234047 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d426944a-6e3b-4177-a1b5-64b6626c28ef-cert\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.235010 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.234972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" event={"ID":"aa754c7e-32e9-4b85-a87c-31f2aa76b87c","Type":"ContainerStarted","Data":"0d83bc2cbfb3ca348db4d358c6fb3eb0e145329ff295e2d3bc817592f8b627a9"} Jul 08 21:41:03.335018 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.334987 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqqbz\" (UniqueName: \"kubernetes.io/projected/d426944a-6e3b-4177-a1b5-64b6626c28ef-kube-api-access-dqqbz\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.335018 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.335023 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d426944a-6e3b-4177-a1b5-64b6626c28ef-metrics-cert\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.335299 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.335045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d426944a-6e3b-4177-a1b5-64b6626c28ef-manager-config\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.335299 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.335064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d426944a-6e3b-4177-a1b5-64b6626c28ef-cert\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.335818 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.335781 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d426944a-6e3b-4177-a1b5-64b6626c28ef-manager-config\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.337660 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.337637 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d426944a-6e3b-4177-a1b5-64b6626c28ef-cert\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.337660 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.337655 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d426944a-6e3b-4177-a1b5-64b6626c28ef-metrics-cert\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.343932 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.343907 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqqbz\" (UniqueName: \"kubernetes.io/projected/d426944a-6e3b-4177-a1b5-64b6626c28ef-kube-api-access-dqqbz\") pod \"lws-controller-manager-5947469944-fx6bz\" (UID: \"d426944a-6e3b-4177-a1b5-64b6626c28ef\") " pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.529616 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.529523 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:03.668963 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:03.668939 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5947469944-fx6bz"] Jul 08 21:41:03.671465 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:41:03.671424 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd426944a_6e3b_4177_a1b5_64b6626c28ef.slice/crio-488e1ef5a4d985277f7fac5fccc5aad3619e0e5488f12b2fdc3f4670b3af2910 WatchSource:0}: Error finding container 488e1ef5a4d985277f7fac5fccc5aad3619e0e5488f12b2fdc3f4670b3af2910: Status 404 returned error can't find the container with id 488e1ef5a4d985277f7fac5fccc5aad3619e0e5488f12b2fdc3f4670b3af2910 Jul 08 21:41:04.241053 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:04.241010 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" event={"ID":"d426944a-6e3b-4177-a1b5-64b6626c28ef","Type":"ContainerStarted","Data":"488e1ef5a4d985277f7fac5fccc5aad3619e0e5488f12b2fdc3f4670b3af2910"} Jul 08 21:41:07.240774 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:07.240735 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 21:41:07.241052 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:07.240813 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 21:41:07.241052 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:07.240843 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 21:41:08.266355 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.266319 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" event={"ID":"aa754c7e-32e9-4b85-a87c-31f2aa76b87c","Type":"ContainerStarted","Data":"73b798ad3038752e7a9201f27163ed22a1a3d2fb4f0e77e430b76f047605d329"} Jul 08 21:41:08.267749 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.267725 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" event={"ID":"d426944a-6e3b-4177-a1b5-64b6626c28ef","Type":"ContainerStarted","Data":"d20299082b95a725fbd066ae2cc05fe8e5072d4e58421d063975e328909b8366"} Jul 08 21:41:08.267856 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.267767 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:08.287912 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.287863 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" podStartSLOduration=2.022806778 podStartE2EDuration="6.287848248s" totalImagesPullingTime="4.26504147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:41:08.286511164 +0000 UTC m=+491.508521366" watchObservedRunningTime="2026-07-08 21:41:08.287848248 +0000 UTC m=+491.509858448" Jul 08 21:41:08.305476 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.305430 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" podStartSLOduration=1.68455645 podStartE2EDuration="5.305411362s" totalImagesPullingTime="3.620854912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:41:08.303609042 +0000 UTC m=+491.525619243" watchObservedRunningTime="2026-07-08 21:41:08.305411362 +0000 UTC m=+491.527421565" Jul 08 21:41:08.785215 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.785150 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:08.790350 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:08.790327 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:09.271680 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:09.271640 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:09.272583 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:09.272565 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llcm" Jul 08 21:41:11.226578 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:11.226546 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7wdjv" Jul 08 21:41:19.273601 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:19.273570 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5947469944-fx6bz" Jul 08 21:41:24.295272 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.295210 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d5ffdb76-n2qdp" podUID="6f55df5c-5107-4b8c-922e-ac5db54c9908" containerName="console" containerID="cri-o://50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161" gracePeriod=15 Jul 08 21:41:24.548111 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.548052 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:41:24.627650 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627616 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8m7q\" (UniqueName: \"kubernetes.io/projected/6f55df5c-5107-4b8c-922e-ac5db54c9908-kube-api-access-d8m7q\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.627650 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627650 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-oauth-config\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.627894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627689 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-trusted-ca-bundle\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.627894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627723 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-config\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.627894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627757 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-service-ca\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.627894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627835 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-oauth-serving-cert\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.627894 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.627881 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-serving-cert\") pod \"6f55df5c-5107-4b8c-922e-ac5db54c9908\" (UID: \"6f55df5c-5107-4b8c-922e-ac5db54c9908\") " Jul 08 21:41:24.628252 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.628207 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:41:24.628252 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.628225 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-config" (OuterVolumeSpecName: "console-config") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:41:24.628383 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.628315 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:41:24.628383 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.628318 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:41:24.630237 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.630214 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:41:24.630480 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.630453 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:41:24.630480 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.630463 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f55df5c-5107-4b8c-922e-ac5db54c9908-kube-api-access-d8m7q" (OuterVolumeSpecName: "kube-api-access-d8m7q") pod "6f55df5c-5107-4b8c-922e-ac5db54c9908" (UID: "6f55df5c-5107-4b8c-922e-ac5db54c9908"). InnerVolumeSpecName "kube-api-access-d8m7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:41:24.728728 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728697 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-oauth-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:24.728728 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728726 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-serving-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:24.728728 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728735 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8m7q\" (UniqueName: \"kubernetes.io/projected/6f55df5c-5107-4b8c-922e-ac5db54c9908-kube-api-access-d8m7q\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:24.728965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728743 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-oauth-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:24.728965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728752 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-trusted-ca-bundle\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:24.728965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728760 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-console-config\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:24.728965 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:24.728772 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f55df5c-5107-4b8c-922e-ac5db54c9908-service-ca\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:41:25.339293 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.339256 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55df5c-5107-4b8c-922e-ac5db54c9908" containerID="50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161" exitCode=0 Jul 08 21:41:25.339725 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.339298 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ffdb76-n2qdp" Jul 08 21:41:25.339725 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.339303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ffdb76-n2qdp" event={"ID":"6f55df5c-5107-4b8c-922e-ac5db54c9908","Type":"ContainerDied","Data":"50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161"} Jul 08 21:41:25.339725 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.339355 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ffdb76-n2qdp" event={"ID":"6f55df5c-5107-4b8c-922e-ac5db54c9908","Type":"ContainerDied","Data":"417d9a68797e1e1cd0085abb100e93fc8d14f058e530296fc5ed719500990216"} Jul 08 21:41:25.339725 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.339376 2563 scope.go:117] "RemoveContainer" containerID="50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161" Jul 08 21:41:25.347663 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.347632 2563 scope.go:117] "RemoveContainer" containerID="50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161" Jul 08 21:41:25.347937 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:41:25.347914 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161\": container with ID starting with 50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161 not found: ID does not exist" containerID="50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161" Jul 08 21:41:25.348027 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.347942 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161"} err="failed to get container status \"50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161\": rpc error: code = NotFound desc = could not find container \"50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161\": container with ID starting with 50f01081c5477b7fab370ae8a7332f1ef5b710b1b30db2dadd8aee4cb7b78161 not found: ID does not exist" Jul 08 21:41:25.354616 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.354596 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d5ffdb76-n2qdp"] Jul 08 21:41:25.356657 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:25.356638 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d5ffdb76-n2qdp"] Jul 08 21:41:27.250696 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:41:27.250655 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f55df5c-5107-4b8c-922e-ac5db54c9908" path="/var/lib/kubelet/pods/6f55df5c-5107-4b8c-922e-ac5db54c9908/volumes" Jul 08 21:43:38.050798 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.050717 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-tndx8"] Jul 08 21:43:38.053162 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.051032 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55df5c-5107-4b8c-922e-ac5db54c9908" containerName="console" Jul 08 21:43:38.053162 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.051045 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55df5c-5107-4b8c-922e-ac5db54c9908" containerName="console" Jul 08 21:43:38.053162 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.051095 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55df5c-5107-4b8c-922e-ac5db54c9908" containerName="console" Jul 08 21:43:38.054066 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.054050 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:38.056205 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.056184 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r5wxh\"" Jul 08 21:43:38.056205 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.056201 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 21:43:38.056806 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.056779 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 21:43:38.061334 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.061308 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tndx8"] Jul 08 21:43:38.173917 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.173877 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5rh\" (UniqueName: \"kubernetes.io/projected/5f85b45c-568a-4a6d-81c1-3a06814285e2-kube-api-access-2p5rh\") pod \"authorino-58cd955675-tndx8\" (UID: \"5f85b45c-568a-4a6d-81c1-3a06814285e2\") " pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:38.274743 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.274708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5rh\" (UniqueName: \"kubernetes.io/projected/5f85b45c-568a-4a6d-81c1-3a06814285e2-kube-api-access-2p5rh\") pod \"authorino-58cd955675-tndx8\" (UID: \"5f85b45c-568a-4a6d-81c1-3a06814285e2\") " pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:38.284937 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.284905 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5rh\" (UniqueName: \"kubernetes.io/projected/5f85b45c-568a-4a6d-81c1-3a06814285e2-kube-api-access-2p5rh\") pod \"authorino-58cd955675-tndx8\" (UID: \"5f85b45c-568a-4a6d-81c1-3a06814285e2\") " pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:38.363905 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.363873 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:38.493669 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.493639 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tndx8"] Jul 08 21:43:38.495655 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:43:38.495635 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f85b45c_568a_4a6d_81c1_3a06814285e2.slice/crio-d3c51f141d7b32cd303b8a17de3e2b1eb8ec8e15de5db782777ca4d33efec2bf WatchSource:0}: Error finding container d3c51f141d7b32cd303b8a17de3e2b1eb8ec8e15de5db782777ca4d33efec2bf: Status 404 returned error can't find the container with id d3c51f141d7b32cd303b8a17de3e2b1eb8ec8e15de5db782777ca4d33efec2bf Jul 08 21:43:38.890601 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:38.890570 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tndx8" event={"ID":"5f85b45c-568a-4a6d-81c1-3a06814285e2","Type":"ContainerStarted","Data":"d3c51f141d7b32cd303b8a17de3e2b1eb8ec8e15de5db782777ca4d33efec2bf"} Jul 08 21:43:41.906203 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:41.906162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tndx8" event={"ID":"5f85b45c-568a-4a6d-81c1-3a06814285e2","Type":"ContainerStarted","Data":"4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d"} Jul 08 21:43:41.919540 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:41.919479 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-tndx8" podStartSLOduration=1.014823508 podStartE2EDuration="3.919429116s" totalImagesPullingTime="2.904605608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:43:41.919021216 +0000 UTC m=+645.141031419" watchObservedRunningTime="2026-07-08 21:43:41.919429116 +0000 UTC m=+645.141439321" Jul 08 21:43:43.308041 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:43.307998 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tndx8"] Jul 08 21:43:43.914077 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:43.914018 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-tndx8" podUID="5f85b45c-568a-4a6d-81c1-3a06814285e2" containerName="authorino" containerID="cri-o://4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d" gracePeriod=30 Jul 08 21:43:44.166438 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.166377 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:44.228053 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.228023 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p5rh\" (UniqueName: \"kubernetes.io/projected/5f85b45c-568a-4a6d-81c1-3a06814285e2-kube-api-access-2p5rh\") pod \"5f85b45c-568a-4a6d-81c1-3a06814285e2\" (UID: \"5f85b45c-568a-4a6d-81c1-3a06814285e2\") " Jul 08 21:43:44.230366 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.230338 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f85b45c-568a-4a6d-81c1-3a06814285e2-kube-api-access-2p5rh" (OuterVolumeSpecName: "kube-api-access-2p5rh") pod "5f85b45c-568a-4a6d-81c1-3a06814285e2" (UID: "5f85b45c-568a-4a6d-81c1-3a06814285e2"). InnerVolumeSpecName "kube-api-access-2p5rh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:43:44.328868 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.328831 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p5rh\" (UniqueName: \"kubernetes.io/projected/5f85b45c-568a-4a6d-81c1-3a06814285e2-kube-api-access-2p5rh\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:43:44.919640 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.919600 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f85b45c-568a-4a6d-81c1-3a06814285e2" containerID="4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d" exitCode=0 Jul 08 21:43:44.919640 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.919643 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tndx8" Jul 08 21:43:44.919901 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.919644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tndx8" event={"ID":"5f85b45c-568a-4a6d-81c1-3a06814285e2","Type":"ContainerDied","Data":"4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d"} Jul 08 21:43:44.919901 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.919693 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tndx8" event={"ID":"5f85b45c-568a-4a6d-81c1-3a06814285e2","Type":"ContainerDied","Data":"d3c51f141d7b32cd303b8a17de3e2b1eb8ec8e15de5db782777ca4d33efec2bf"} Jul 08 21:43:44.919901 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.919715 2563 scope.go:117] "RemoveContainer" containerID="4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d" Jul 08 21:43:44.928072 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.928055 2563 scope.go:117] "RemoveContainer" containerID="4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d" Jul 08 21:43:44.928350 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:43:44.928331 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d\": container with ID starting with 4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d not found: ID does not exist" containerID="4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d" Jul 08 21:43:44.928404 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.928359 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d"} err="failed to get container status \"4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d\": rpc error: code = NotFound desc = could not find container \"4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d\": container with ID starting with 4f924e20c93b7b95d731d4b987cd7c970bce182c2fe912521083647b76d3c62d not found: ID does not exist" Jul 08 21:43:44.938758 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.938736 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tndx8"] Jul 08 21:43:44.943976 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:44.943955 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-tndx8"] Jul 08 21:43:45.250075 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:43:45.249994 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f85b45c-568a-4a6d-81c1-3a06814285e2" path="/var/lib/kubelet/pods/5f85b45c-568a-4a6d-81c1-3a06814285e2/volumes" Jul 08 21:44:05.748808 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.748772 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g27ct"] Jul 08 21:44:05.749206 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.749081 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f85b45c-568a-4a6d-81c1-3a06814285e2" containerName="authorino" Jul 08 21:44:05.749206 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.749092 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f85b45c-568a-4a6d-81c1-3a06814285e2" containerName="authorino" Jul 08 21:44:05.749206 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.749164 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f85b45c-568a-4a6d-81c1-3a06814285e2" containerName="authorino" Jul 08 21:44:05.753176 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.753159 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:05.755476 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.755454 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 21:44:05.755591 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.755501 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 21:44:05.756158 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.756144 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r5wxh\"" Jul 08 21:44:05.759782 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.759754 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g27ct"] Jul 08 21:44:05.926178 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.926116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrgv\" (UniqueName: \"kubernetes.io/projected/55cb6ad8-1c47-4370-8650-22c3d81ea893-kube-api-access-twrgv\") pod \"authorino-5d4b67c446-g27ct\" (UID: \"55cb6ad8-1c47-4370-8650-22c3d81ea893\") " pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:05.980239 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:05.980202 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g27ct"] Jul 08 21:44:05.980464 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:44:05.980446 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-twrgv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-g27ct" podUID="55cb6ad8-1c47-4370-8650-22c3d81ea893" Jul 08 21:44:06.004006 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.003921 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-576ff9bb5d-v2tmg"] Jul 08 21:44:06.007107 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.007085 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:06.013302 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.013274 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:06.017357 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.017336 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-576ff9bb5d-v2tmg"] Jul 08 21:44:06.021661 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.021643 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:06.027022 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.027004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrgv\" (UniqueName: \"kubernetes.io/projected/55cb6ad8-1c47-4370-8650-22c3d81ea893-kube-api-access-twrgv\") pod \"authorino-5d4b67c446-g27ct\" (UID: \"55cb6ad8-1c47-4370-8650-22c3d81ea893\") " pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:06.042475 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.042442 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrgv\" (UniqueName: \"kubernetes.io/projected/55cb6ad8-1c47-4370-8650-22c3d81ea893-kube-api-access-twrgv\") pod \"authorino-5d4b67c446-g27ct\" (UID: \"55cb6ad8-1c47-4370-8650-22c3d81ea893\") " pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:06.127684 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.127647 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twrgv\" (UniqueName: \"kubernetes.io/projected/55cb6ad8-1c47-4370-8650-22c3d81ea893-kube-api-access-twrgv\") pod \"55cb6ad8-1c47-4370-8650-22c3d81ea893\" (UID: \"55cb6ad8-1c47-4370-8650-22c3d81ea893\") " Jul 08 21:44:06.127857 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.127809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghtlb\" (UniqueName: \"kubernetes.io/projected/7fe65939-ce73-41f6-a8fe-b477898899f2-kube-api-access-ghtlb\") pod \"authorino-576ff9bb5d-v2tmg\" (UID: \"7fe65939-ce73-41f6-a8fe-b477898899f2\") " pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:06.129851 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.129829 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55cb6ad8-1c47-4370-8650-22c3d81ea893-kube-api-access-twrgv" (OuterVolumeSpecName: "kube-api-access-twrgv") pod "55cb6ad8-1c47-4370-8650-22c3d81ea893" (UID: "55cb6ad8-1c47-4370-8650-22c3d81ea893"). InnerVolumeSpecName "kube-api-access-twrgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:44:06.229102 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.229064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghtlb\" (UniqueName: \"kubernetes.io/projected/7fe65939-ce73-41f6-a8fe-b477898899f2-kube-api-access-ghtlb\") pod \"authorino-576ff9bb5d-v2tmg\" (UID: \"7fe65939-ce73-41f6-a8fe-b477898899f2\") " pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:06.229304 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.229217 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twrgv\" (UniqueName: \"kubernetes.io/projected/55cb6ad8-1c47-4370-8650-22c3d81ea893-kube-api-access-twrgv\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:44:06.236998 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.236976 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghtlb\" (UniqueName: \"kubernetes.io/projected/7fe65939-ce73-41f6-a8fe-b477898899f2-kube-api-access-ghtlb\") pod \"authorino-576ff9bb5d-v2tmg\" (UID: \"7fe65939-ce73-41f6-a8fe-b477898899f2\") " pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:06.280210 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.280104 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-576ff9bb5d-v2tmg"] Jul 08 21:44:06.280429 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.280416 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:06.306209 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.306182 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7889dbcc6b-47fxn"] Jul 08 21:44:06.309693 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.309665 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.311656 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.311633 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 21:44:06.316533 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.316510 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7889dbcc6b-47fxn"] Jul 08 21:44:06.414354 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.414333 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-576ff9bb5d-v2tmg"] Jul 08 21:44:06.417021 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:44:06.416993 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fe65939_ce73_41f6_a8fe_b477898899f2.slice/crio-37676d287c6b4e3418118578b070b799b65004bb3aa3fd6792623b9792b1cdc9 WatchSource:0}: Error finding container 37676d287c6b4e3418118578b070b799b65004bb3aa3fd6792623b9792b1cdc9: Status 404 returned error can't find the container with id 37676d287c6b4e3418118578b070b799b65004bb3aa3fd6792623b9792b1cdc9 Jul 08 21:44:06.431290 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.431265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9147d611-6054-467e-b01b-1b45d08276dd-tls-cert\") pod \"authorino-7889dbcc6b-47fxn\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.431374 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.431315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxjds\" (UniqueName: \"kubernetes.io/projected/9147d611-6054-467e-b01b-1b45d08276dd-kube-api-access-lxjds\") pod \"authorino-7889dbcc6b-47fxn\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.532611 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.532536 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxjds\" (UniqueName: \"kubernetes.io/projected/9147d611-6054-467e-b01b-1b45d08276dd-kube-api-access-lxjds\") pod \"authorino-7889dbcc6b-47fxn\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.532611 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.532599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9147d611-6054-467e-b01b-1b45d08276dd-tls-cert\") pod \"authorino-7889dbcc6b-47fxn\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.535026 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.535000 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9147d611-6054-467e-b01b-1b45d08276dd-tls-cert\") pod \"authorino-7889dbcc6b-47fxn\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.541354 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.541322 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxjds\" (UniqueName: \"kubernetes.io/projected/9147d611-6054-467e-b01b-1b45d08276dd-kube-api-access-lxjds\") pod \"authorino-7889dbcc6b-47fxn\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.620825 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.620787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:44:06.751202 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:06.751178 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7889dbcc6b-47fxn"] Jul 08 21:44:06.753446 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:44:06.753416 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9147d611_6054_467e_b01b_1b45d08276dd.slice/crio-78bceda837a301bca226d22f006547904f08a02a3f0f9c52b2a67d47d2cc9f0e WatchSource:0}: Error finding container 78bceda837a301bca226d22f006547904f08a02a3f0f9c52b2a67d47d2cc9f0e: Status 404 returned error can't find the container with id 78bceda837a301bca226d22f006547904f08a02a3f0f9c52b2a67d47d2cc9f0e Jul 08 21:44:07.018891 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.018852 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" event={"ID":"9147d611-6054-467e-b01b-1b45d08276dd","Type":"ContainerStarted","Data":"78bceda837a301bca226d22f006547904f08a02a3f0f9c52b2a67d47d2cc9f0e"} Jul 08 21:44:07.020407 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.020374 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" event={"ID":"7fe65939-ce73-41f6-a8fe-b477898899f2","Type":"ContainerStarted","Data":"50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b"} Jul 08 21:44:07.020541 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.020419 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" event={"ID":"7fe65939-ce73-41f6-a8fe-b477898899f2","Type":"ContainerStarted","Data":"37676d287c6b4e3418118578b070b799b65004bb3aa3fd6792623b9792b1cdc9"} Jul 08 21:44:07.020541 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.020416 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" podUID="7fe65939-ce73-41f6-a8fe-b477898899f2" containerName="authorino" containerID="cri-o://50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b" gracePeriod=30 Jul 08 21:44:07.020541 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.020445 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g27ct" Jul 08 21:44:07.035966 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.035921 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" podStartSLOduration=1.711558726 podStartE2EDuration="2.035907137s" totalImagesPullingTime="324.348411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:44:07.034253983 +0000 UTC m=+670.256264184" watchObservedRunningTime="2026-07-08 21:44:07.035907137 +0000 UTC m=+670.257917339" Jul 08 21:44:07.055321 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.055253 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g27ct"] Jul 08 21:44:07.058711 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.058682 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g27ct"] Jul 08 21:44:07.251824 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.251799 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55cb6ad8-1c47-4370-8650-22c3d81ea893" path="/var/lib/kubelet/pods/55cb6ad8-1c47-4370-8650-22c3d81ea893/volumes" Jul 08 21:44:07.268793 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.268765 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:07.440895 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.440860 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghtlb\" (UniqueName: \"kubernetes.io/projected/7fe65939-ce73-41f6-a8fe-b477898899f2-kube-api-access-ghtlb\") pod \"7fe65939-ce73-41f6-a8fe-b477898899f2\" (UID: \"7fe65939-ce73-41f6-a8fe-b477898899f2\") " Jul 08 21:44:07.443090 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.443063 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fe65939-ce73-41f6-a8fe-b477898899f2-kube-api-access-ghtlb" (OuterVolumeSpecName: "kube-api-access-ghtlb") pod "7fe65939-ce73-41f6-a8fe-b477898899f2" (UID: "7fe65939-ce73-41f6-a8fe-b477898899f2"). InnerVolumeSpecName "kube-api-access-ghtlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:44:07.542164 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:07.542112 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghtlb\" (UniqueName: \"kubernetes.io/projected/7fe65939-ce73-41f6-a8fe-b477898899f2-kube-api-access-ghtlb\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:44:08.026338 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.026300 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" event={"ID":"9147d611-6054-467e-b01b-1b45d08276dd","Type":"ContainerStarted","Data":"71c62184a8b08747ec868f1af8b4fccfe11dcd2bf04382989f189e9eb17c9365"} Jul 08 21:44:08.027592 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.027572 2563 generic.go:356] "Generic (PLEG): container finished" podID="7fe65939-ce73-41f6-a8fe-b477898899f2" containerID="50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b" exitCode=0 Jul 08 21:44:08.027691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.027605 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" Jul 08 21:44:08.027691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.027634 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" event={"ID":"7fe65939-ce73-41f6-a8fe-b477898899f2","Type":"ContainerDied","Data":"50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b"} Jul 08 21:44:08.027691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.027662 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576ff9bb5d-v2tmg" event={"ID":"7fe65939-ce73-41f6-a8fe-b477898899f2","Type":"ContainerDied","Data":"37676d287c6b4e3418118578b070b799b65004bb3aa3fd6792623b9792b1cdc9"} Jul 08 21:44:08.027691 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.027679 2563 scope.go:117] "RemoveContainer" containerID="50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b" Jul 08 21:44:08.037622 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.037597 2563 scope.go:117] "RemoveContainer" containerID="50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b" Jul 08 21:44:08.037915 ip-10-0-130-8 kubenswrapper[2563]: E0708 21:44:08.037897 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b\": container with ID starting with 50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b not found: ID does not exist" containerID="50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b" Jul 08 21:44:08.037979 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.037923 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b"} err="failed to get container status \"50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b\": rpc error: code = NotFound desc = could not find container \"50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b\": container with ID starting with 50b1b84ab6b80ec049f85735bb366be921eb6b1e3533f02230e8e24d64ba416b not found: ID does not exist" Jul 08 21:44:08.041494 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.041446 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" podStartSLOduration=0.848085808 podStartE2EDuration="2.041426725s" totalImagesPullingTime="1.193340917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:44:08.039809857 +0000 UTC m=+671.261820060" watchObservedRunningTime="2026-07-08 21:44:08.041426725 +0000 UTC m=+671.263436927" Jul 08 21:44:08.054619 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.054585 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-576ff9bb5d-v2tmg"] Jul 08 21:44:08.060492 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:08.060454 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-576ff9bb5d-v2tmg"] Jul 08 21:44:09.250109 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:44:09.250075 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fe65939-ce73-41f6-a8fe-b477898899f2" path="/var/lib/kubelet/pods/7fe65939-ce73-41f6-a8fe-b477898899f2/volumes" Jul 08 21:45:43.353450 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.353356 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q"] Jul 08 21:45:43.353911 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.353665 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fe65939-ce73-41f6-a8fe-b477898899f2" containerName="authorino" Jul 08 21:45:43.353911 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.353677 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fe65939-ce73-41f6-a8fe-b477898899f2" containerName="authorino" Jul 08 21:45:43.353911 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.353745 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fe65939-ce73-41f6-a8fe-b477898899f2" containerName="authorino" Jul 08 21:45:43.356843 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.356827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.359702 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.359670 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 21:45:43.359702 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.359691 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 21:45:43.359903 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.359673 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tr7vh\"" Jul 08 21:45:43.359903 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.359671 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 21:45:43.366695 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.366666 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q"] Jul 08 21:45:43.474414 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.474378 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.474414 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.474421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.474665 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.474442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrmvr\" (UniqueName: \"kubernetes.io/projected/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-kube-api-access-hrmvr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.474665 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.474499 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.474665 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.474522 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.474665 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.474536 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575297 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575297 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575515 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575515 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575361 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575515 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575515 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575412 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrmvr\" (UniqueName: \"kubernetes.io/projected/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-kube-api-access-hrmvr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575711 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575664 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575711 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575685 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.575806 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.575742 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.577802 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.577781 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.578080 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.578065 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.583105 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.583078 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrmvr\" (UniqueName: \"kubernetes.io/projected/e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5-kube-api-access-hrmvr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q\" (UID: \"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.667173 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.667106 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:43.796106 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.795955 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q"] Jul 08 21:45:43.798808 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:45:43.798783 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39faf2e_8d81_4ad1_b9b2_c6a0a4faddd5.slice/crio-025124ced0c6ac3e23ebf839f2d1d644982b966c4819b6245198b936e062c492 WatchSource:0}: Error finding container 025124ced0c6ac3e23ebf839f2d1d644982b966c4819b6245198b936e062c492: Status 404 returned error can't find the container with id 025124ced0c6ac3e23ebf839f2d1d644982b966c4819b6245198b936e062c492 Jul 08 21:45:43.800075 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:43.800059 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:45:44.440823 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:44.440783 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" event={"ID":"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5","Type":"ContainerStarted","Data":"025124ced0c6ac3e23ebf839f2d1d644982b966c4819b6245198b936e062c492"} Jul 08 21:45:45.446417 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:45.446378 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" event={"ID":"e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5","Type":"ContainerStarted","Data":"bf3e9de5a08e968fc9ccb62dee7b631aab5c5cd9157f14251e4a793290ad39d8"} Jul 08 21:45:45.465320 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:45.465261 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" podStartSLOduration=1.46716734 podStartE2EDuration="2.465245972s" totalImagesPullingTime="998.078632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:45:45.463528105 +0000 UTC m=+768.685538309" watchObservedRunningTime="2026-07-08 21:45:45.465245972 +0000 UTC m=+768.687256173" Jul 08 21:45:53.667494 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:53.667456 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:53.667895 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:53.667527 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:53.680235 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:53.680200 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:45:54.494759 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:45:54.494722 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q" Jul 08 21:46:10.962286 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:10.962253 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c"] Jul 08 21:46:10.965432 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:10.965413 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:10.968218 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:10.968198 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 21:46:10.981467 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:10.981438 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c"] Jul 08 21:46:11.021355 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.021319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.021564 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.021366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.021564 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.021395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.021564 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.021426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hn8s\" (UniqueName: \"kubernetes.io/projected/0a73855c-d84a-4bd0-a403-4ace2d7dc523-kube-api-access-4hn8s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.021564 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.021518 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.021564 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.021540 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a73855c-d84a-4bd0-a403-4ace2d7dc523-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122154 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122421 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a73855c-d84a-4bd0-a403-4ace2d7dc523-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122421 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122421 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122231 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122421 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122421 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122293 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hn8s\" (UniqueName: \"kubernetes.io/projected/0a73855c-d84a-4bd0-a403-4ace2d7dc523-kube-api-access-4hn8s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122705 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122680 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122760 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122712 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.122810 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.122793 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.124548 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.124528 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a73855c-d84a-4bd0-a403-4ace2d7dc523-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.124794 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.124778 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a73855c-d84a-4bd0-a403-4ace2d7dc523-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.140838 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.140808 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hn8s\" (UniqueName: \"kubernetes.io/projected/0a73855c-d84a-4bd0-a403-4ace2d7dc523-kube-api-access-4hn8s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-44j7c\" (UID: \"0a73855c-d84a-4bd0-a403-4ace2d7dc523\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.275470 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.275384 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:11.418101 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.418073 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c"] Jul 08 21:46:11.420490 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:46:11.420462 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a73855c_d84a_4bd0_a403_4ace2d7dc523.slice/crio-59e522f731e4b8e6f81e17633cbebd203a7e805ccd7dc2859f0990521bfa24d4 WatchSource:0}: Error finding container 59e522f731e4b8e6f81e17633cbebd203a7e805ccd7dc2859f0990521bfa24d4: Status 404 returned error can't find the container with id 59e522f731e4b8e6f81e17633cbebd203a7e805ccd7dc2859f0990521bfa24d4 Jul 08 21:46:11.555177 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:11.555063 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" event={"ID":"0a73855c-d84a-4bd0-a403-4ace2d7dc523","Type":"ContainerStarted","Data":"59e522f731e4b8e6f81e17633cbebd203a7e805ccd7dc2859f0990521bfa24d4"} Jul 08 21:46:12.561292 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:12.561252 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" event={"ID":"0a73855c-d84a-4bd0-a403-4ace2d7dc523","Type":"ContainerStarted","Data":"59fb600dfc9d83f40861359e7fe9a016d97d122bd8e9fdb168ba84e3d6bd0808"} Jul 08 21:46:12.580410 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:12.580357 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" podStartSLOduration=2.413854439 podStartE2EDuration="2.580339157s" totalImagesPullingTime="166.484718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:46:12.578323792 +0000 UTC m=+795.800333994" watchObservedRunningTime="2026-07-08 21:46:12.580339157 +0000 UTC m=+795.802349358" Jul 08 21:46:18.960038 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:18.959958 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc"] Jul 08 21:46:18.965704 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:18.965686 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:18.968510 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:18.968488 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 21:46:18.973169 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:18.973147 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc"] Jul 08 21:46:19.092060 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.092026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.092252 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.092078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.092252 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.092205 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d20aded-57a5-4c9d-801f-767730260df9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.092252 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.092236 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn288\" (UniqueName: \"kubernetes.io/projected/0d20aded-57a5-4c9d-801f-767730260df9-kube-api-access-pn288\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.092385 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.092288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.092385 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.092322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.193566 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.193522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.193771 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.193584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.193771 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.193700 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.193771 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.193747 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d20aded-57a5-4c9d-801f-767730260df9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.193934 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.193795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn288\" (UniqueName: \"kubernetes.io/projected/0d20aded-57a5-4c9d-801f-767730260df9-kube-api-access-pn288\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.193934 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.193825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.194073 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.194051 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.194161 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.194147 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.194207 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.194180 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.195920 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.195889 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d20aded-57a5-4c9d-801f-767730260df9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.196263 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.196246 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d20aded-57a5-4c9d-801f-767730260df9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.202067 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.202043 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn288\" (UniqueName: \"kubernetes.io/projected/0d20aded-57a5-4c9d-801f-767730260df9-kube-api-access-pn288\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc\" (UID: \"0d20aded-57a5-4c9d-801f-767730260df9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.275483 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.275401 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:19.411018 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.410984 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc"] Jul 08 21:46:19.414110 ip-10-0-130-8 kubenswrapper[2563]: W0708 21:46:19.414085 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d20aded_57a5_4c9d_801f_767730260df9.slice/crio-3a457b7d15fbf72f6ca7c306ab5a8ffc7816fa7fe619ff6bdcecb068911ee770 WatchSource:0}: Error finding container 3a457b7d15fbf72f6ca7c306ab5a8ffc7816fa7fe619ff6bdcecb068911ee770: Status 404 returned error can't find the container with id 3a457b7d15fbf72f6ca7c306ab5a8ffc7816fa7fe619ff6bdcecb068911ee770 Jul 08 21:46:19.592442 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:19.592363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" event={"ID":"0d20aded-57a5-4c9d-801f-767730260df9","Type":"ContainerStarted","Data":"3a457b7d15fbf72f6ca7c306ab5a8ffc7816fa7fe619ff6bdcecb068911ee770"} Jul 08 21:46:20.597990 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:20.597947 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" event={"ID":"0d20aded-57a5-4c9d-801f-767730260df9","Type":"ContainerStarted","Data":"f295df4c23199497aadff0021b1beb208e7027a6b9c3eebb26077cef4786901a"} Jul 08 21:46:20.616854 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:20.616801 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" podStartSLOduration=2.413903843 podStartE2EDuration="2.616782347s" totalImagesPullingTime="202.878504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:46:20.61540545 +0000 UTC m=+803.837415651" watchObservedRunningTime="2026-07-08 21:46:20.616782347 +0000 UTC m=+803.838792548" Jul 08 21:46:21.275531 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:21.275483 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:21.275775 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:21.275558 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:21.288812 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:21.288776 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:21.618441 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:21.618410 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-44j7c" Jul 08 21:46:29.276273 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:29.276239 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:29.276676 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:29.276318 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:29.288538 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:29.288509 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:46:29.647247 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:46:29.647219 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc" Jul 08 21:47:48.818198 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:48.816564 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7889dbcc6b-47fxn"] Jul 08 21:47:48.818198 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:48.816777 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" podUID="9147d611-6054-467e-b01b-1b45d08276dd" containerName="authorino" containerID="cri-o://71c62184a8b08747ec868f1af8b4fccfe11dcd2bf04382989f189e9eb17c9365" gracePeriod=30 Jul 08 21:47:48.985456 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:48.985413 2563 generic.go:356] "Generic (PLEG): container finished" podID="9147d611-6054-467e-b01b-1b45d08276dd" containerID="71c62184a8b08747ec868f1af8b4fccfe11dcd2bf04382989f189e9eb17c9365" exitCode=0 Jul 08 21:47:48.985618 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:48.985480 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" event={"ID":"9147d611-6054-467e-b01b-1b45d08276dd","Type":"ContainerDied","Data":"71c62184a8b08747ec868f1af8b4fccfe11dcd2bf04382989f189e9eb17c9365"} Jul 08 21:47:49.076362 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.076298 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:47:49.180769 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.180732 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9147d611-6054-467e-b01b-1b45d08276dd-tls-cert\") pod \"9147d611-6054-467e-b01b-1b45d08276dd\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " Jul 08 21:47:49.180954 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.180790 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxjds\" (UniqueName: \"kubernetes.io/projected/9147d611-6054-467e-b01b-1b45d08276dd-kube-api-access-lxjds\") pod \"9147d611-6054-467e-b01b-1b45d08276dd\" (UID: \"9147d611-6054-467e-b01b-1b45d08276dd\") " Jul 08 21:47:49.182888 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.182864 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9147d611-6054-467e-b01b-1b45d08276dd-kube-api-access-lxjds" (OuterVolumeSpecName: "kube-api-access-lxjds") pod "9147d611-6054-467e-b01b-1b45d08276dd" (UID: "9147d611-6054-467e-b01b-1b45d08276dd"). InnerVolumeSpecName "kube-api-access-lxjds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:47:49.191090 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.191056 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9147d611-6054-467e-b01b-1b45d08276dd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9147d611-6054-467e-b01b-1b45d08276dd" (UID: "9147d611-6054-467e-b01b-1b45d08276dd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:47:49.281766 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.281734 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9147d611-6054-467e-b01b-1b45d08276dd-tls-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:47:49.281766 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.281765 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxjds\" (UniqueName: \"kubernetes.io/projected/9147d611-6054-467e-b01b-1b45d08276dd-kube-api-access-lxjds\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 21:47:49.991339 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.991305 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" event={"ID":"9147d611-6054-467e-b01b-1b45d08276dd","Type":"ContainerDied","Data":"78bceda837a301bca226d22f006547904f08a02a3f0f9c52b2a67d47d2cc9f0e"} Jul 08 21:47:49.991339 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.991335 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7889dbcc6b-47fxn" Jul 08 21:47:49.991820 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:49.991360 2563 scope.go:117] "RemoveContainer" containerID="71c62184a8b08747ec868f1af8b4fccfe11dcd2bf04382989f189e9eb17c9365" Jul 08 21:47:50.007131 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:50.007094 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7889dbcc6b-47fxn"] Jul 08 21:47:50.009781 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:50.009757 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7889dbcc6b-47fxn"] Jul 08 21:47:51.250536 ip-10-0-130-8 kubenswrapper[2563]: I0708 21:47:51.250507 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9147d611-6054-467e-b01b-1b45d08276dd" path="/var/lib/kubelet/pods/9147d611-6054-467e-b01b-1b45d08276dd/volumes" Jul 08 22:01:10.573625 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.573591 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6"] Jul 08 22:01:10.576076 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.573926 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9147d611-6054-467e-b01b-1b45d08276dd" containerName="authorino" Jul 08 22:01:10.576076 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.573946 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9147d611-6054-467e-b01b-1b45d08276dd" containerName="authorino" Jul 08 22:01:10.576076 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.574014 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9147d611-6054-467e-b01b-1b45d08276dd" containerName="authorino" Jul 08 22:01:10.577032 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.577015 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.581726 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.581699 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8e25db08-openshift-default-dockercfg-dbxlg\"" Jul 08 22:01:10.593562 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.593532 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6"] Jul 08 22:01:10.676400 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676356 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-socket\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676605 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb762355-8cd5-4533-8c37-e8ce7610743b-istiod-ca-cert\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676605 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676510 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-certs\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676605 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676545 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-podinfo\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676605 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-credential-socket\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676810 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-envoy\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676810 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676659 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-token\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676810 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676714 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-data\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.676810 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.676742 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4wzs\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-kube-api-access-z4wzs\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778292 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-credential-socket\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778292 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-envoy\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778555 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778321 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-token\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778555 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778345 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-data\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778555 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778375 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4wzs\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-kube-api-access-z4wzs\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778555 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-socket\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778555 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb762355-8cd5-4533-8c37-e8ce7610743b-istiod-ca-cert\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778971 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-certs\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.778971 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778766 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-data\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.779170 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778852 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-socket\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.779170 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.778726 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-credential-socket\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.779170 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.779034 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-podinfo\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.779170 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.779160 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-certs\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.779392 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.779238 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb762355-8cd5-4533-8c37-e8ce7610743b-istiod-ca-cert\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.780729 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.780710 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-envoy\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.781626 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.781603 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-podinfo\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.786237 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.786212 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-token\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.786332 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.786279 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4wzs\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-kube-api-access-z4wzs\") pod \"e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:10.887939 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:10.887898 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:11.028557 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.028535 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6"] Jul 08 22:01:11.031295 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:01:11.031266 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb762355_8cd5_4533_8c37_e8ce7610743b.slice/crio-d00df1b1fa1ef1e5013458e1c5763a7196ba2885ea994e4bce1c40b52d569841 WatchSource:0}: Error finding container d00df1b1fa1ef1e5013458e1c5763a7196ba2885ea994e4bce1c40b52d569841: Status 404 returned error can't find the container with id d00df1b1fa1ef1e5013458e1c5763a7196ba2885ea994e4bce1c40b52d569841 Jul 08 22:01:11.033234 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.033201 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:01:11.033327 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.033267 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:01:11.033327 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.033299 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:01:11.354518 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.354416 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" event={"ID":"cb762355-8cd5-4533-8c37-e8ce7610743b","Type":"ContainerStarted","Data":"39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d"} Jul 08 22:01:11.354518 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.354511 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" event={"ID":"cb762355-8cd5-4533-8c37-e8ce7610743b","Type":"ContainerStarted","Data":"d00df1b1fa1ef1e5013458e1c5763a7196ba2885ea994e4bce1c40b52d569841"} Jul 08 22:01:11.376290 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.376230 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" podStartSLOduration=1.376211058 podStartE2EDuration="1.376211058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:01:11.375428617 +0000 UTC m=+1694.597438832" watchObservedRunningTime="2026-07-08 22:01:11.376211058 +0000 UTC m=+1694.598221261" Jul 08 22:01:11.888599 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:11.888566 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:12.893710 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:12.893680 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:13.364821 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:13.364790 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:13.365747 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:13.365729 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:16.941283 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:16.941235 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6"] Jul 08 22:01:16.941806 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:16.941591 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" podUID="cb762355-8cd5-4533-8c37-e8ce7610743b" containerName="istio-proxy" containerID="cri-o://39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d" gracePeriod=30 Jul 08 22:01:23.596481 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.596411 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:23.695554 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695496 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb762355-8cd5-4533-8c37-e8ce7610743b-istiod-ca-cert\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695740 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695604 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-data\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695740 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695630 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4wzs\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-kube-api-access-z4wzs\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695740 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695690 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-socket\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695740 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695716 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-credential-socket\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695946 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695751 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-podinfo\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695946 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695779 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-token\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695946 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695823 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-certs\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.695946 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.695856 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-envoy\") pod \"cb762355-8cd5-4533-8c37-e8ce7610743b\" (UID: \"cb762355-8cd5-4533-8c37-e8ce7610743b\") " Jul 08 22:01:23.696171 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696014 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb762355-8cd5-4533-8c37-e8ce7610743b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:01:23.696171 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696033 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:23.696171 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696008 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:23.696332 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696249 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:23.696332 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696253 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb762355-8cd5-4533-8c37-e8ce7610743b-istiod-ca-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.696332 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696292 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.696332 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696309 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-credential-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.696814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.696775 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:23.698166 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.698138 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:23.698281 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.698212 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:01:23.698281 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.698244 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-kube-api-access-z4wzs" (OuterVolumeSpecName: "kube-api-access-z4wzs") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "kube-api-access-z4wzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:01:23.698359 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.698301 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb762355-8cd5-4533-8c37-e8ce7610743b" (UID: "cb762355-8cd5-4533-8c37-e8ce7610743b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 22:01:23.797143 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.797072 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-podinfo\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.797143 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.797113 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-token\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.797143 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.797134 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-workload-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.797143 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.797142 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-envoy\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.797143 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.797150 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb762355-8cd5-4533-8c37-e8ce7610743b-istio-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:23.797143 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:23.797158 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4wzs\" (UniqueName: \"kubernetes.io/projected/cb762355-8cd5-4533-8c37-e8ce7610743b-kube-api-access-z4wzs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:01:24.411747 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.411712 2563 generic.go:356] "Generic (PLEG): container finished" podID="cb762355-8cd5-4533-8c37-e8ce7610743b" containerID="39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d" exitCode=0 Jul 08 22:01:24.411944 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.411763 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" Jul 08 22:01:24.411944 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.411812 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" event={"ID":"cb762355-8cd5-4533-8c37-e8ce7610743b","Type":"ContainerDied","Data":"39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d"} Jul 08 22:01:24.411944 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.411870 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6" event={"ID":"cb762355-8cd5-4533-8c37-e8ce7610743b","Type":"ContainerDied","Data":"d00df1b1fa1ef1e5013458e1c5763a7196ba2885ea994e4bce1c40b52d569841"} Jul 08 22:01:24.411944 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.411898 2563 scope.go:117] "RemoveContainer" containerID="39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d" Jul 08 22:01:24.421066 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.421048 2563 scope.go:117] "RemoveContainer" containerID="39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d" Jul 08 22:01:24.421381 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:01:24.421360 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d\": container with ID starting with 39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d not found: ID does not exist" containerID="39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d" Jul 08 22:01:24.421438 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.421390 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d"} err="failed to get container status \"39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d\": rpc error: code = NotFound desc = could not find container \"39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d\": container with ID starting with 39c2e4890902415d1146960f1b4482c83dddab8c4b71fca883c10b66b4cb810d not found: ID does not exist" Jul 08 22:01:24.436724 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.436686 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6"] Jul 08 22:01:24.442567 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:24.442533 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8e25db08-openshift-default-5d79ddd87-m77v6"] Jul 08 22:01:25.251101 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:01:25.251066 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb762355-8cd5-4533-8c37-e8ce7610743b" path="/var/lib/kubelet/pods/cb762355-8cd5-4533-8c37-e8ce7610743b/volumes" Jul 08 22:04:23.945904 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.945866 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks"] Jul 08 22:04:23.946351 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.946213 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb762355-8cd5-4533-8c37-e8ce7610743b" containerName="istio-proxy" Jul 08 22:04:23.946351 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.946225 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb762355-8cd5-4533-8c37-e8ce7610743b" containerName="istio-proxy" Jul 08 22:04:23.946351 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.946277 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb762355-8cd5-4533-8c37-e8ce7610743b" containerName="istio-proxy" Jul 08 22:04:23.949174 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.949152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:23.951483 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.951463 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5efb71b0\"/\"kube-root-ca.crt\"" Jul 08 22:04:23.951598 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.951491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5efb71b0\"/\"openshift-service-ca.crt\"" Jul 08 22:04:23.952361 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.952339 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5efb71b0\"/\"default-dockercfg-mspgq\"" Jul 08 22:04:23.952468 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.952366 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5efb71b0\"/\"e2e-lifecycle-model-5efb71b0-kserve-self-signed-certs\"" Jul 08 22:04:23.959336 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:23.959311 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks"] Jul 08 22:04:24.121835 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.121799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-tmp-dir\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.122014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.121845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-home\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.122014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.121948 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/084c969c-ba10-47b8-8560-9351feb9399c-tls-certs\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.122089 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.122050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-dshm\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.122089 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.122079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvkl4\" (UniqueName: \"kubernetes.io/projected/084c969c-ba10-47b8-8560-9351feb9399c-kube-api-access-hvkl4\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.122188 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.122175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-model-cache\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.222967 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.222859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-dshm\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.222967 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.222907 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvkl4\" (UniqueName: \"kubernetes.io/projected/084c969c-ba10-47b8-8560-9351feb9399c-kube-api-access-hvkl4\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.222967 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.222953 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-model-cache\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.223308 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.222999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-tmp-dir\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.223308 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.223191 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-home\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.223308 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.223263 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/084c969c-ba10-47b8-8560-9351feb9399c-tls-certs\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.223460 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.223423 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-model-cache\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.223516 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.223475 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-tmp-dir\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.223516 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.223506 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-home\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.225374 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.225352 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-dshm\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.225778 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.225760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/084c969c-ba10-47b8-8560-9351feb9399c-tls-certs\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.230518 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.230492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvkl4\" (UniqueName: \"kubernetes.io/projected/084c969c-ba10-47b8-8560-9351feb9399c-kube-api-access-hvkl4\") pod \"e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.259483 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.259449 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:24.399564 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:24.399539 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks"] Jul 08 22:04:24.402169 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:04:24.402141 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod084c969c_ba10_47b8_8560_9351feb9399c.slice/crio-b12596d2d3423e6d5ba185dba965f202958f90d7ecae4e7fd5b312e0dc37fed6 WatchSource:0}: Error finding container b12596d2d3423e6d5ba185dba965f202958f90d7ecae4e7fd5b312e0dc37fed6: Status 404 returned error can't find the container with id b12596d2d3423e6d5ba185dba965f202958f90d7ecae4e7fd5b312e0dc37fed6 Jul 08 22:04:25.175023 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:25.174974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" event={"ID":"084c969c-ba10-47b8-8560-9351feb9399c","Type":"ContainerStarted","Data":"6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf"} Jul 08 22:04:25.175023 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:25.175025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" event={"ID":"084c969c-ba10-47b8-8560-9351feb9399c","Type":"ContainerStarted","Data":"b12596d2d3423e6d5ba185dba965f202958f90d7ecae4e7fd5b312e0dc37fed6"} Jul 08 22:04:25.191205 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:25.191157 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" podStartSLOduration=2.191139642 podStartE2EDuration="2.191139642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:04:25.189499314 +0000 UTC m=+1888.411509518" watchObservedRunningTime="2026-07-08 22:04:25.191139642 +0000 UTC m=+1888.413149833" Jul 08 22:04:34.259778 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:34.259732 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:34.260212 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:34.259809 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:34.272613 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:34.272590 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:35.229243 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:35.229214 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:04:54.318746 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:04:54.318708 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks"] Jul 08 22:05:09.339986 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.339940 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" podUID="084c969c-ba10-47b8-8560-9351feb9399c" containerName="main" containerID="cri-o://6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf" gracePeriod=45 Jul 08 22:05:09.595872 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.595807 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:05:09.726967 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.726935 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/084c969c-ba10-47b8-8560-9351feb9399c-tls-certs\") pod \"084c969c-ba10-47b8-8560-9351feb9399c\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " Jul 08 22:05:09.727173 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.726999 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvkl4\" (UniqueName: \"kubernetes.io/projected/084c969c-ba10-47b8-8560-9351feb9399c-kube-api-access-hvkl4\") pod \"084c969c-ba10-47b8-8560-9351feb9399c\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " Jul 08 22:05:09.727173 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727034 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-tmp-dir\") pod \"084c969c-ba10-47b8-8560-9351feb9399c\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " Jul 08 22:05:09.727173 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727161 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-home\") pod \"084c969c-ba10-47b8-8560-9351feb9399c\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " Jul 08 22:05:09.727344 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727235 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-model-cache\") pod \"084c969c-ba10-47b8-8560-9351feb9399c\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " Jul 08 22:05:09.727344 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727264 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-dshm\") pod \"084c969c-ba10-47b8-8560-9351feb9399c\" (UID: \"084c969c-ba10-47b8-8560-9351feb9399c\") " Jul 08 22:05:09.727344 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727289 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "084c969c-ba10-47b8-8560-9351feb9399c" (UID: "084c969c-ba10-47b8-8560-9351feb9399c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:05:09.727344 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727313 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-home" (OuterVolumeSpecName: "home") pod "084c969c-ba10-47b8-8560-9351feb9399c" (UID: "084c969c-ba10-47b8-8560-9351feb9399c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:05:09.727554 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727478 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-model-cache" (OuterVolumeSpecName: "model-cache") pod "084c969c-ba10-47b8-8560-9351feb9399c" (UID: "084c969c-ba10-47b8-8560-9351feb9399c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:05:09.727765 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727695 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-model-cache\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:05:09.727765 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727714 2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-tmp-dir\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:05:09.727765 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.727722 2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-home\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:05:09.729423 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.729397 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-dshm" (OuterVolumeSpecName: "dshm") pod "084c969c-ba10-47b8-8560-9351feb9399c" (UID: "084c969c-ba10-47b8-8560-9351feb9399c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:05:09.729423 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.729401 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/084c969c-ba10-47b8-8560-9351feb9399c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "084c969c-ba10-47b8-8560-9351feb9399c" (UID: "084c969c-ba10-47b8-8560-9351feb9399c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:05:09.729537 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.729470 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/084c969c-ba10-47b8-8560-9351feb9399c-kube-api-access-hvkl4" (OuterVolumeSpecName: "kube-api-access-hvkl4") pod "084c969c-ba10-47b8-8560-9351feb9399c" (UID: "084c969c-ba10-47b8-8560-9351feb9399c"). InnerVolumeSpecName "kube-api-access-hvkl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:05:09.828963 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.828911 2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/084c969c-ba10-47b8-8560-9351feb9399c-dshm\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:05:09.828963 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.828954 2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/084c969c-ba10-47b8-8560-9351feb9399c-tls-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:05:09.828963 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:09.828963 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvkl4\" (UniqueName: \"kubernetes.io/projected/084c969c-ba10-47b8-8560-9351feb9399c-kube-api-access-hvkl4\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:05:10.365377 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.365340 2563 generic.go:356] "Generic (PLEG): container finished" podID="084c969c-ba10-47b8-8560-9351feb9399c" containerID="6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf" exitCode=0 Jul 08 22:05:10.365377 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.365381 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" Jul 08 22:05:10.365829 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.365386 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" event={"ID":"084c969c-ba10-47b8-8560-9351feb9399c","Type":"ContainerDied","Data":"6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf"} Jul 08 22:05:10.365829 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.365425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks" event={"ID":"084c969c-ba10-47b8-8560-9351feb9399c","Type":"ContainerDied","Data":"b12596d2d3423e6d5ba185dba965f202958f90d7ecae4e7fd5b312e0dc37fed6"} Jul 08 22:05:10.365829 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.365448 2563 scope.go:117] "RemoveContainer" containerID="6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf" Jul 08 22:05:10.374191 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.374170 2563 scope.go:117] "RemoveContainer" containerID="6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf" Jul 08 22:05:10.374498 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:05:10.374472 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf\": container with ID starting with 6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf not found: ID does not exist" containerID="6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf" Jul 08 22:05:10.374583 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.374510 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf"} err="failed to get container status \"6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf\": rpc error: code = NotFound desc = could not find container \"6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf\": container with ID starting with 6a7c139b8b5972af31fd255936b08829f7e9cf960d98909c5646b815c5e47cbf not found: ID does not exist" Jul 08 22:05:10.384681 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.384649 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks"] Jul 08 22:05:10.388002 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:10.387974 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5efb71b0/e2e-lifecycle-model-5efb71b0-kserve-654965d867-frbks"] Jul 08 22:05:11.250567 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:05:11.250535 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="084c969c-ba10-47b8-8560-9351feb9399c" path="/var/lib/kubelet/pods/084c969c-ba10-47b8-8560-9351feb9399c/volumes" Jul 08 22:06:40.026921 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.026881 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm"] Jul 08 22:06:40.029697 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.027344 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="084c969c-ba10-47b8-8560-9351feb9399c" containerName="main" Jul 08 22:06:40.029697 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.027358 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="084c969c-ba10-47b8-8560-9351feb9399c" containerName="main" Jul 08 22:06:40.029697 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.027451 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="084c969c-ba10-47b8-8560-9351feb9399c" containerName="main" Jul 08 22:06:40.030433 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.030414 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.032622 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.032595 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3eddaa-openshift-default-dockercfg-b5r2r\"" Jul 08 22:06:40.042470 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.042445 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm"] Jul 08 22:06:40.072516 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fksjc\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-kube-api-access-fksjc\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072704 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istiod-ca-cert\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072704 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072581 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-socket\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072704 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072599 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-envoy\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072704 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-podinfo\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072704 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-token\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072969 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-credential-socket\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072969 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072766 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-data\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.072969 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.072794 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-certs\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173236 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173187 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-socket\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173236 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-envoy\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173524 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173264 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-podinfo\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173524 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-token\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173524 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173299 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-credential-socket\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173524 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173350 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-data\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173524 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173384 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-certs\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173524 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173488 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fksjc\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-kube-api-access-fksjc\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173873 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istiod-ca-cert\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173873 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173655 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-socket\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.173873 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.173777 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-credential-socket\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.174238 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.174024 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-certs\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.174238 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.174224 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istiod-ca-cert\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.174348 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.174238 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-data\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.175696 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.175674 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-envoy\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.175902 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.175883 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-podinfo\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.180765 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.180744 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-token\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.180862 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.180819 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fksjc\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-kube-api-access-fksjc\") pod \"e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.341456 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.341359 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:40.485582 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.485543 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm"] Jul 08 22:06:40.489489 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:06:40.489455 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefeb02d3_5602_4fe4_b015_eeb0c81aad69.slice/crio-4d1d96bfa989e8ae244b96acc57e4d0644c8bf9b88dc0dc17e532049d3695f24 WatchSource:0}: Error finding container 4d1d96bfa989e8ae244b96acc57e4d0644c8bf9b88dc0dc17e532049d3695f24: Status 404 returned error can't find the container with id 4d1d96bfa989e8ae244b96acc57e4d0644c8bf9b88dc0dc17e532049d3695f24 Jul 08 22:06:40.491686 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.491646 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:06:40.491763 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.491736 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:06:40.491808 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.491777 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:06:40.746739 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.746704 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" event={"ID":"efeb02d3-5602-4fe4-b015-eeb0c81aad69","Type":"ContainerStarted","Data":"57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc"} Jul 08 22:06:40.746900 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.746761 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" event={"ID":"efeb02d3-5602-4fe4-b015-eeb0c81aad69","Type":"ContainerStarted","Data":"4d1d96bfa989e8ae244b96acc57e4d0644c8bf9b88dc0dc17e532049d3695f24"} Jul 08 22:06:40.766653 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:40.766594 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podStartSLOduration=0.766575564 podStartE2EDuration="766.575564ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:06:40.765779075 +0000 UTC m=+2023.987789275" watchObservedRunningTime="2026-07-08 22:06:40.766575564 +0000 UTC m=+2023.988585765" Jul 08 22:06:41.342533 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:41.342495 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:41.347367 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:41.347343 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:41.751396 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:41.751359 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:41.753500 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:41.753477 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:06:55.050030 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.049990 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f"] Jul 08 22:06:55.055604 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.055571 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.059718 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.059687 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2c94a2\"/\"default-dockercfg-kvsdj\"" Jul 08 22:06:55.059872 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.059842 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2c94a2\"/\"kube-root-ca.crt\"" Jul 08 22:06:55.059941 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.059912 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2c94a2\"/\"openshift-service-ca.crt\"" Jul 08 22:06:55.060202 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.060154 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2c94a2\"/\"test-model-2c94a2-kserve-self-signed-certs\"" Jul 08 22:06:55.060603 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.060573 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f"] Jul 08 22:06:55.106033 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.106002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc6fj\" (UniqueName: \"kubernetes.io/projected/6f79339e-14d2-4877-aad1-2ed0f5e0850c-kube-api-access-dc6fj\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.106033 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.106048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-home\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.106294 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.106066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tls-certs\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.106294 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.106174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-dshm\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.106294 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.106211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tmp-dir\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.106294 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.106232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-model-cache\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.206813 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.206779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-dshm\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.206821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tmp-dir\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.206845 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-model-cache\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.206912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc6fj\" (UniqueName: \"kubernetes.io/projected/6f79339e-14d2-4877-aad1-2ed0f5e0850c-kube-api-access-dc6fj\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.206953 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-home\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.206975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tls-certs\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207316 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.207301 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tmp-dir\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207411 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.207392 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-model-cache\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.207464 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.207451 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-home\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.209393 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.209357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-dshm\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.209528 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.209513 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tls-certs\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.215573 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.215547 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc6fj\" (UniqueName: \"kubernetes.io/projected/6f79339e-14d2-4877-aad1-2ed0f5e0850c-kube-api-access-dc6fj\") pod \"test-model-2c94a2-kserve-696cc84bdc-77m5f\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.367834 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.367791 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:06:55.521942 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.521853 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f"] Jul 08 22:06:55.531569 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:06:55.531534 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f79339e_14d2_4877_aad1_2ed0f5e0850c.slice/crio-934d5ff3cba343e348213dabda50713a4b9e17cc122111bdb8f718cb889e6b93 WatchSource:0}: Error finding container 934d5ff3cba343e348213dabda50713a4b9e17cc122111bdb8f718cb889e6b93: Status 404 returned error can't find the container with id 934d5ff3cba343e348213dabda50713a4b9e17cc122111bdb8f718cb889e6b93 Jul 08 22:06:55.811305 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.811202 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" event={"ID":"6f79339e-14d2-4877-aad1-2ed0f5e0850c","Type":"ContainerStarted","Data":"e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb"} Jul 08 22:06:55.811305 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.811258 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" event={"ID":"6f79339e-14d2-4877-aad1-2ed0f5e0850c","Type":"ContainerStarted","Data":"934d5ff3cba343e348213dabda50713a4b9e17cc122111bdb8f718cb889e6b93"} Jul 08 22:06:55.828601 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:55.828555 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" podStartSLOduration=0.828539246 podStartE2EDuration="828.539246ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:06:55.827083679 +0000 UTC m=+2039.049093880" watchObservedRunningTime="2026-07-08 22:06:55.828539246 +0000 UTC m=+2039.050549446" Jul 08 22:06:57.793322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:57.793277 2563 patch_prober.go:28] interesting pod/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 22:06:57.793708 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:57.793349 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 22:06:58.826413 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:58.826377 2563 generic.go:356] "Generic (PLEG): container finished" podID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerID="57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc" exitCode=0 Jul 08 22:06:58.826995 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:58.826443 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" event={"ID":"efeb02d3-5602-4fe4-b015-eeb0c81aad69","Type":"ContainerDied","Data":"57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc"} Jul 08 22:06:58.826995 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:58.826833 2563 scope.go:117] "RemoveContainer" containerID="57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc" Jul 08 22:06:58.827871 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:58.827831 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:06:58.827965 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:58.827899 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:06:58.827965 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:58.827932 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:06:59.836313 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:06:59.836270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" event={"ID":"efeb02d3-5602-4fe4-b015-eeb0c81aad69","Type":"ContainerStarted","Data":"deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41"} Jul 08 22:07:00.341870 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:00.341820 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:07:00.342112 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:00.341975 2563 patch_prober.go:28] interesting pod/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 22:07:00.342112 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:00.342025 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 22:07:01.342321 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:01.342271 2563 patch_prober.go:28] interesting pod/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 22:07:01.342873 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:01.342358 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 22:07:02.342426 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:02.342379 2563 patch_prober.go:28] interesting pod/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 22:07:02.342975 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:02.342451 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 22:07:03.342896 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:03.342846 2563 patch_prober.go:28] interesting pod/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 08 22:07:03.343453 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:03.342934 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 08 22:07:04.345643 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:04.345612 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:07:04.346055 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:04.345979 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:07:04.346704 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:04.346683 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:07:05.368847 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:05.368806 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:07:05.369265 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:05.369066 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:07:05.381366 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:05.381340 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:07:05.884153 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:05.884101 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:07:34.852796 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.852765 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8"] Jul 08 22:07:34.856016 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.855997 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.858196 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.858116 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3eddaa\"/\"kube-root-ca.crt\"" Jul 08 22:07:34.858806 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.858790 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3eddaa\"/\"default-dockercfg-rrbb7\"" Jul 08 22:07:34.858964 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.858947 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3eddaa\"/\"openshift-service-ca.crt\"" Jul 08 22:07:34.859027 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.858959 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3eddaa\"/\"test-model-3eddaa-kserve-self-signed-certs\"" Jul 08 22:07:34.867744 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.867719 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8"] Jul 08 22:07:34.868105 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.868083 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-home\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.868230 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.868116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-tmp-dir\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.868230 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.868165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-model-cache\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.868322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.868272 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d46367-bcdc-41a7-88a1-a58b09cad076-tls-certs\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.868322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.868315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89l8s\" (UniqueName: \"kubernetes.io/projected/e4d46367-bcdc-41a7-88a1-a58b09cad076-kube-api-access-89l8s\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.868422 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.868406 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-dshm\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969400 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-model-cache\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969591 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969421 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d46367-bcdc-41a7-88a1-a58b09cad076-tls-certs\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969591 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969458 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89l8s\" (UniqueName: \"kubernetes.io/projected/e4d46367-bcdc-41a7-88a1-a58b09cad076-kube-api-access-89l8s\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969591 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969507 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-dshm\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969591 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969539 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-home\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969591 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-tmp-dir\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969859 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969784 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-model-cache\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.969915 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969872 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-tmp-dir\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.970004 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.969981 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-home\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.971956 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.971935 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-dshm\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.972237 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.972221 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d46367-bcdc-41a7-88a1-a58b09cad076-tls-certs\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:34.977336 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:34.977318 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89l8s\" (UniqueName: \"kubernetes.io/projected/e4d46367-bcdc-41a7-88a1-a58b09cad076-kube-api-access-89l8s\") pod \"test-model-3eddaa-kserve-5c64cdddbf-6gtc8\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:35.166427 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:35.166386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:35.300457 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:35.300431 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8"] Jul 08 22:07:35.303022 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:07:35.302991 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4d46367_bcdc_41a7_88a1_a58b09cad076.slice/crio-cdc79deda6269817f9d321c38904c1455f9ca99ab91b78f8cbf23300457c69b6 WatchSource:0}: Error finding container cdc79deda6269817f9d321c38904c1455f9ca99ab91b78f8cbf23300457c69b6: Status 404 returned error can't find the container with id cdc79deda6269817f9d321c38904c1455f9ca99ab91b78f8cbf23300457c69b6 Jul 08 22:07:36.004658 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:36.004622 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" event={"ID":"e4d46367-bcdc-41a7-88a1-a58b09cad076","Type":"ContainerStarted","Data":"275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196"} Jul 08 22:07:36.005062 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:36.004667 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" event={"ID":"e4d46367-bcdc-41a7-88a1-a58b09cad076","Type":"ContainerStarted","Data":"cdc79deda6269817f9d321c38904c1455f9ca99ab91b78f8cbf23300457c69b6"} Jul 08 22:07:36.021936 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:36.021864 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" podStartSLOduration=2.021843286 podStartE2EDuration="2.021843286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:07:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:07:36.021394345 +0000 UTC m=+2079.243404606" watchObservedRunningTime="2026-07-08 22:07:36.021843286 +0000 UTC m=+2079.243853487" Jul 08 22:07:45.167250 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:45.167192 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:45.167673 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:45.167270 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:45.179688 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:45.179661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:46.059172 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:46.059119 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:07:55.666699 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:55.666666 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f"] Jul 08 22:07:56.555275 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:07:56.555108 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8"] Jul 08 22:08:10.687631 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:10.687592 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" podUID="6f79339e-14d2-4877-aad1-2ed0f5e0850c" containerName="main" containerID="cri-o://e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb" gracePeriod=45 Jul 08 22:08:10.935657 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:10.935632 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:08:11.004699 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004611 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tls-certs\") pod \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " Jul 08 22:08:11.004699 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004651 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-home\") pod \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " Jul 08 22:08:11.004699 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004694 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-model-cache\") pod \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " Jul 08 22:08:11.004980 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004792 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc6fj\" (UniqueName: \"kubernetes.io/projected/6f79339e-14d2-4877-aad1-2ed0f5e0850c-kube-api-access-dc6fj\") pod \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " Jul 08 22:08:11.004980 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004830 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-dshm\") pod \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " Jul 08 22:08:11.004980 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004938 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tmp-dir\") pod \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\" (UID: \"6f79339e-14d2-4877-aad1-2ed0f5e0850c\") " Jul 08 22:08:11.004980 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004943 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-home" (OuterVolumeSpecName: "home") pod "6f79339e-14d2-4877-aad1-2ed0f5e0850c" (UID: "6f79339e-14d2-4877-aad1-2ed0f5e0850c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.004980 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.004964 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f79339e-14d2-4877-aad1-2ed0f5e0850c" (UID: "6f79339e-14d2-4877-aad1-2ed0f5e0850c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.005269 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.005231 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f79339e-14d2-4877-aad1-2ed0f5e0850c" (UID: "6f79339e-14d2-4877-aad1-2ed0f5e0850c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.005309 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.005274 2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-home\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:11.005309 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.005295 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-model-cache\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:11.007141 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.007097 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-dshm" (OuterVolumeSpecName: "dshm") pod "6f79339e-14d2-4877-aad1-2ed0f5e0850c" (UID: "6f79339e-14d2-4877-aad1-2ed0f5e0850c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.007242 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.007174 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f79339e-14d2-4877-aad1-2ed0f5e0850c" (UID: "6f79339e-14d2-4877-aad1-2ed0f5e0850c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:08:11.007242 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.007180 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f79339e-14d2-4877-aad1-2ed0f5e0850c-kube-api-access-dc6fj" (OuterVolumeSpecName: "kube-api-access-dc6fj") pod "6f79339e-14d2-4877-aad1-2ed0f5e0850c" (UID: "6f79339e-14d2-4877-aad1-2ed0f5e0850c"). InnerVolumeSpecName "kube-api-access-dc6fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:08:11.106070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.106032 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc6fj\" (UniqueName: \"kubernetes.io/projected/6f79339e-14d2-4877-aad1-2ed0f5e0850c-kube-api-access-dc6fj\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:11.106070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.106065 2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-dshm\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:11.106070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.106076 2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tmp-dir\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:11.106337 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.106084 2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f79339e-14d2-4877-aad1-2ed0f5e0850c-tls-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:11.156627 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.156595 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f79339e-14d2-4877-aad1-2ed0f5e0850c" containerID="e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb" exitCode=0 Jul 08 22:08:11.156832 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.156638 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" Jul 08 22:08:11.156832 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.156689 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" event={"ID":"6f79339e-14d2-4877-aad1-2ed0f5e0850c","Type":"ContainerDied","Data":"e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb"} Jul 08 22:08:11.156832 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.156740 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f" event={"ID":"6f79339e-14d2-4877-aad1-2ed0f5e0850c","Type":"ContainerDied","Data":"934d5ff3cba343e348213dabda50713a4b9e17cc122111bdb8f718cb889e6b93"} Jul 08 22:08:11.156832 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.156763 2563 scope.go:117] "RemoveContainer" containerID="e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb" Jul 08 22:08:11.166114 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.166096 2563 scope.go:117] "RemoveContainer" containerID="e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb" Jul 08 22:08:11.166416 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:08:11.166398 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb\": container with ID starting with e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb not found: ID does not exist" containerID="e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb" Jul 08 22:08:11.166466 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.166424 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb"} err="failed to get container status \"e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb\": rpc error: code = NotFound desc = could not find container \"e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb\": container with ID starting with e1bb752389f8aa06f43c6a992547116f12baf7f42dca698b0c9e253814d9eefb not found: ID does not exist" Jul 08 22:08:11.177088 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.177063 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f"] Jul 08 22:08:11.180720 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.180694 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-2c94a2/test-model-2c94a2-kserve-696cc84bdc-77m5f"] Jul 08 22:08:11.250334 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.250299 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f79339e-14d2-4877-aad1-2ed0f5e0850c" path="/var/lib/kubelet/pods/6f79339e-14d2-4877-aad1-2ed0f5e0850c/volumes" Jul 08 22:08:11.576287 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.576225 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" podUID="e4d46367-bcdc-41a7-88a1-a58b09cad076" containerName="main" containerID="cri-o://275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196" gracePeriod=45 Jul 08 22:08:11.823935 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.823909 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:08:11.912210 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912171 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d46367-bcdc-41a7-88a1-a58b09cad076-tls-certs\") pod \"e4d46367-bcdc-41a7-88a1-a58b09cad076\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " Jul 08 22:08:11.912415 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912236 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-dshm\") pod \"e4d46367-bcdc-41a7-88a1-a58b09cad076\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " Jul 08 22:08:11.912415 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912307 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-model-cache\") pod \"e4d46367-bcdc-41a7-88a1-a58b09cad076\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " Jul 08 22:08:11.912415 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912327 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89l8s\" (UniqueName: \"kubernetes.io/projected/e4d46367-bcdc-41a7-88a1-a58b09cad076-kube-api-access-89l8s\") pod \"e4d46367-bcdc-41a7-88a1-a58b09cad076\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " Jul 08 22:08:11.912415 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912355 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-home\") pod \"e4d46367-bcdc-41a7-88a1-a58b09cad076\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " Jul 08 22:08:11.912415 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912381 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-tmp-dir\") pod \"e4d46367-bcdc-41a7-88a1-a58b09cad076\" (UID: \"e4d46367-bcdc-41a7-88a1-a58b09cad076\") " Jul 08 22:08:11.912672 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912616 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4d46367-bcdc-41a7-88a1-a58b09cad076" (UID: "e4d46367-bcdc-41a7-88a1-a58b09cad076"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.912672 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912641 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-home" (OuterVolumeSpecName: "home") pod "e4d46367-bcdc-41a7-88a1-a58b09cad076" (UID: "e4d46367-bcdc-41a7-88a1-a58b09cad076"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.912776 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.912729 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4d46367-bcdc-41a7-88a1-a58b09cad076" (UID: "e4d46367-bcdc-41a7-88a1-a58b09cad076"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.914620 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.914597 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-dshm" (OuterVolumeSpecName: "dshm") pod "e4d46367-bcdc-41a7-88a1-a58b09cad076" (UID: "e4d46367-bcdc-41a7-88a1-a58b09cad076"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:11.914766 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.914657 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4d46367-bcdc-41a7-88a1-a58b09cad076-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4d46367-bcdc-41a7-88a1-a58b09cad076" (UID: "e4d46367-bcdc-41a7-88a1-a58b09cad076"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:08:11.914766 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:11.914698 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d46367-bcdc-41a7-88a1-a58b09cad076-kube-api-access-89l8s" (OuterVolumeSpecName: "kube-api-access-89l8s") pod "e4d46367-bcdc-41a7-88a1-a58b09cad076" (UID: "e4d46367-bcdc-41a7-88a1-a58b09cad076"). InnerVolumeSpecName "kube-api-access-89l8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:08:12.013768 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.013737 2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-dshm\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:12.013768 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.013766 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-model-cache\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:12.013768 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.013777 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89l8s\" (UniqueName: \"kubernetes.io/projected/e4d46367-bcdc-41a7-88a1-a58b09cad076-kube-api-access-89l8s\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:12.014019 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.013787 2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-home\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:12.014019 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.013801 2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d46367-bcdc-41a7-88a1-a58b09cad076-tmp-dir\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:12.014019 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.013812 2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d46367-bcdc-41a7-88a1-a58b09cad076-tls-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:12.162977 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.162887 2563 generic.go:356] "Generic (PLEG): container finished" podID="e4d46367-bcdc-41a7-88a1-a58b09cad076" containerID="275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196" exitCode=0 Jul 08 22:08:12.162977 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.162930 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" Jul 08 22:08:12.162977 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.162960 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" event={"ID":"e4d46367-bcdc-41a7-88a1-a58b09cad076","Type":"ContainerDied","Data":"275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196"} Jul 08 22:08:12.163251 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.163002 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8" event={"ID":"e4d46367-bcdc-41a7-88a1-a58b09cad076","Type":"ContainerDied","Data":"cdc79deda6269817f9d321c38904c1455f9ca99ab91b78f8cbf23300457c69b6"} Jul 08 22:08:12.163251 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.163027 2563 scope.go:117] "RemoveContainer" containerID="275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196" Jul 08 22:08:12.171903 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.171883 2563 scope.go:117] "RemoveContainer" containerID="275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196" Jul 08 22:08:12.172202 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:08:12.172177 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196\": container with ID starting with 275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196 not found: ID does not exist" containerID="275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196" Jul 08 22:08:12.172254 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.172214 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196"} err="failed to get container status \"275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196\": rpc error: code = NotFound desc = could not find container \"275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196\": container with ID starting with 275280cba1bb18f1a62e572e10e0d5f53e9cc318320899c758e7335f9a504196 not found: ID does not exist" Jul 08 22:08:12.183709 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.183679 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8"] Jul 08 22:08:12.187034 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:12.187006 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3eddaa/test-model-3eddaa-kserve-5c64cdddbf-6gtc8"] Jul 08 22:08:13.250230 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:13.250194 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d46367-bcdc-41a7-88a1-a58b09cad076" path="/var/lib/kubelet/pods/e4d46367-bcdc-41a7-88a1-a58b09cad076/volumes" Jul 08 22:08:46.042297 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:46.042262 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm"] Jul 08 22:08:46.042814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:46.042487 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" containerID="cri-o://deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41" gracePeriod=30 Jul 08 22:08:49.348989 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:49.348902 2563 patch_prober.go:28] interesting pod/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 22:08:49.348989 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:49.348972 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 22:08:51.301773 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.301749 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:08:51.340288 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.340193 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" Jul 08 22:08:51.340288 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.340201 2563 generic.go:356] "Generic (PLEG): container finished" podID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerID="deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41" exitCode=0 Jul 08 22:08:51.340288 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.340229 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" event={"ID":"efeb02d3-5602-4fe4-b015-eeb0c81aad69","Type":"ContainerDied","Data":"deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41"} Jul 08 22:08:51.340561 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.340298 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm" event={"ID":"efeb02d3-5602-4fe4-b015-eeb0c81aad69","Type":"ContainerDied","Data":"4d1d96bfa989e8ae244b96acc57e4d0644c8bf9b88dc0dc17e532049d3695f24"} Jul 08 22:08:51.340561 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.340328 2563 scope.go:117] "RemoveContainer" containerID="deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41" Jul 08 22:08:51.348961 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.348939 2563 scope.go:117] "RemoveContainer" containerID="57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc" Jul 08 22:08:51.356881 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.356864 2563 scope.go:117] "RemoveContainer" containerID="deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41" Jul 08 22:08:51.357153 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:08:51.357110 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41\": container with ID starting with deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41 not found: ID does not exist" containerID="deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41" Jul 08 22:08:51.357207 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.357163 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41"} err="failed to get container status \"deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41\": rpc error: code = NotFound desc = could not find container \"deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41\": container with ID starting with deef51bd8c5740feddc288d08dd1d37d36e4f19b438ecf91b800ba2ba4f60e41 not found: ID does not exist" Jul 08 22:08:51.357207 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.357182 2563 scope.go:117] "RemoveContainer" containerID="57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc" Jul 08 22:08:51.357415 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:08:51.357398 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc\": container with ID starting with 57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc not found: ID does not exist" containerID="57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc" Jul 08 22:08:51.357463 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.357421 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc"} err="failed to get container status \"57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc\": rpc error: code = NotFound desc = could not find container \"57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc\": container with ID starting with 57b785286293076d20aa0ca812d908649a29a452317eb17640a3f5c5c1757edc not found: ID does not exist" Jul 08 22:08:51.473400 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473361 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-data\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473649 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473410 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-socket\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473649 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473436 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istiod-ca-cert\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473649 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473457 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-envoy\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473649 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473473 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-token\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473649 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473609 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-certs\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473656 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fksjc\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-kube-api-access-fksjc\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473691 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:51.473923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473710 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-podinfo\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473741 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-credential-socket\") pod \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\" (UID: \"efeb02d3-5602-4fe4-b015-eeb0c81aad69\") " Jul 08 22:08:51.473923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473871 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:51.473923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.473900 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:08:51.474263 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.474097 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:51.474324 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.474265 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.474324 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.474285 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istiod-ca-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.474324 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.474302 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-workload-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.474324 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.474314 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-credential-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.474686 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.474367 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-data" (OuterVolumeSpecName: "istio-data") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:51.476087 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.476060 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-token" (OuterVolumeSpecName: "istio-token") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:08:51.476257 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.476057 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 22:08:51.476435 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.476406 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-kube-api-access-fksjc" (OuterVolumeSpecName: "kube-api-access-fksjc") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "kube-api-access-fksjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:08:51.476435 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.476419 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efeb02d3-5602-4fe4-b015-eeb0c81aad69" (UID: "efeb02d3-5602-4fe4-b015-eeb0c81aad69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:08:51.575146 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.575095 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.575146 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.575152 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-envoy\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.575354 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.575164 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-token\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.575354 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.575175 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fksjc\" (UniqueName: \"kubernetes.io/projected/efeb02d3-5602-4fe4-b015-eeb0c81aad69-kube-api-access-fksjc\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.575354 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.575188 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efeb02d3-5602-4fe4-b015-eeb0c81aad69-istio-podinfo\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:08:51.665669 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.661706 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm"] Jul 08 22:08:51.667373 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:51.667346 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3eddaa-openshift-default-dc96c5f8c-cpjsm"] Jul 08 22:08:53.250903 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:53.250868 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" path="/var/lib/kubelet/pods/efeb02d3-5602-4fe4-b015-eeb0c81aad69/volumes" Jul 08 22:08:54.728710 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.728679 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl"] Jul 08 22:08:54.729070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729011 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f79339e-14d2-4877-aad1-2ed0f5e0850c" containerName="main" Jul 08 22:08:54.729070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729022 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f79339e-14d2-4877-aad1-2ed0f5e0850c" containerName="main" Jul 08 22:08:54.729070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729037 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" Jul 08 22:08:54.729070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729043 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" Jul 08 22:08:54.729070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729055 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d46367-bcdc-41a7-88a1-a58b09cad076" containerName="main" Jul 08 22:08:54.729070 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729060 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d46367-bcdc-41a7-88a1-a58b09cad076" containerName="main" Jul 08 22:08:54.729330 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729114 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f79339e-14d2-4877-aad1-2ed0f5e0850c" containerName="main" Jul 08 22:08:54.729330 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729141 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" Jul 08 22:08:54.729330 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729148 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" Jul 08 22:08:54.729330 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.729153 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d46367-bcdc-41a7-88a1-a58b09cad076" containerName="main" Jul 08 22:08:54.733811 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.733788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.736106 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.736080 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-648128-openshift-default-dockercfg-5dblk\"" Jul 08 22:08:54.744085 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.743722 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl"] Jul 08 22:08:54.802766 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.802729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-socket\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.802766 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.802771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-envoy\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803036 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.802790 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-certs\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803036 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.802809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-istio-token\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803036 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.802946 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-data\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803036 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.802981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69edea7-a78b-49c8-86b4-e991877b8703-istio-podinfo\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803036 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.803015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-credential-socket\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803293 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.803043 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsg4m\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-kube-api-access-rsg4m\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.803293 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.803075 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69edea7-a78b-49c8-86b4-e991877b8703-istiod-ca-cert\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904037 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.903998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsg4m\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-kube-api-access-rsg4m\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904037 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69edea7-a78b-49c8-86b4-e991877b8703-istiod-ca-cert\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-socket\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-envoy\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904162 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-certs\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-istio-token\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904277 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-data\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904322 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904303 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69edea7-a78b-49c8-86b4-e991877b8703-istio-podinfo\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904631 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-credential-socket\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904631 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904573 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-socket\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904762 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904736 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-certs\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.904819 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.904806 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-credential-socket\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.905055 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.905004 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69edea7-a78b-49c8-86b4-e991877b8703-istiod-ca-cert\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.905055 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.905023 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-data\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.906558 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.906537 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-envoy\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.907118 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.907095 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69edea7-a78b-49c8-86b4-e991877b8703-istio-podinfo\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.911770 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.911743 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-istio-token\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:54.911923 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:54.911902 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsg4m\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-kube-api-access-rsg4m\") pod \"e2e-api-b-648128-openshift-default-5d86999747-7mvwl\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:55.049084 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.048983 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:55.194345 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.194308 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl"] Jul 08 22:08:55.197540 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:08:55.197513 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69edea7_a78b_49c8_86b4_e991877b8703.slice/crio-b520e26d71dd17d321e63d61d67953260fce91aaee89e1f8c1583e44596ae5b4 WatchSource:0}: Error finding container b520e26d71dd17d321e63d61d67953260fce91aaee89e1f8c1583e44596ae5b4: Status 404 returned error can't find the container with id b520e26d71dd17d321e63d61d67953260fce91aaee89e1f8c1583e44596ae5b4 Jul 08 22:08:55.200114 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.200038 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:08:55.200297 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.200212 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:08:55.200297 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.200264 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:08:55.361717 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.361677 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" event={"ID":"c69edea7-a78b-49c8-86b4-e991877b8703","Type":"ContainerStarted","Data":"2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4"} Jul 08 22:08:55.361944 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.361750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" event={"ID":"c69edea7-a78b-49c8-86b4-e991877b8703","Type":"ContainerStarted","Data":"b520e26d71dd17d321e63d61d67953260fce91aaee89e1f8c1583e44596ae5b4"} Jul 08 22:08:55.384204 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:55.384118 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podStartSLOduration=1.3840955240000001 podStartE2EDuration="1.384095524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:08:55.381951726 +0000 UTC m=+2158.603961945" watchObservedRunningTime="2026-07-08 22:08:55.384095524 +0000 UTC m=+2158.606105727" Jul 08 22:08:56.049922 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:56.049883 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:56.054755 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:56.054727 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:56.366530 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:56.366499 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:08:56.367600 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:08:56.367581 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:12.234560 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.233545 2563 patch_prober.go:28] interesting pod/e2e-api-b-648128-openshift-default-5d86999747-7mvwl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:60236->10.134.0.40:15021: read: connection reset by peer" start-of-body= Jul 08 22:09:12.234560 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.233614 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:60236->10.134.0.40:15021: read: connection reset by peer" Jul 08 22:09:12.439737 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.439652 2563 generic.go:356] "Generic (PLEG): container finished" podID="c69edea7-a78b-49c8-86b4-e991877b8703" containerID="2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4" exitCode=0 Jul 08 22:09:12.439737 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.439713 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" event={"ID":"c69edea7-a78b-49c8-86b4-e991877b8703","Type":"ContainerDied","Data":"2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4"} Jul 08 22:09:12.440080 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.440067 2563 scope.go:117] "RemoveContainer" containerID="2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4" Jul 08 22:09:12.441048 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.441023 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:12.441146 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.441089 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:12.441195 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:12.441120 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:13.446606 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:13.446567 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" event={"ID":"c69edea7-a78b-49c8-86b4-e991877b8703","Type":"ContainerStarted","Data":"9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b"} Jul 08 22:09:14.049467 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:14.049430 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:14.049700 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:14.049681 2563 patch_prober.go:28] interesting pod/e2e-api-b-648128-openshift-default-5d86999747-7mvwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 22:09:14.049745 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:14.049724 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 22:09:15.049800 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:15.049754 2563 patch_prober.go:28] interesting pod/e2e-api-b-648128-openshift-default-5d86999747-7mvwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 22:09:15.050348 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:15.049846 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 22:09:16.049671 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:16.049638 2563 patch_prober.go:28] interesting pod/e2e-api-b-648128-openshift-default-5d86999747-7mvwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 22:09:16.049886 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:16.049725 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 22:09:17.050299 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:17.050259 2563 patch_prober.go:28] interesting pod/e2e-api-b-648128-openshift-default-5d86999747-7mvwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 22:09:17.050788 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:17.050320 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 22:09:18.049673 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:18.049635 2563 patch_prober.go:28] interesting pod/e2e-api-b-648128-openshift-default-5d86999747-7mvwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 08 22:09:18.049853 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:18.049701 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 08 22:09:19.053076 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:19.053043 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:19.053555 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:19.053339 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:19.054093 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:19.054076 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:23.084213 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:23.084179 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl"] Jul 08 22:09:23.084709 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:23.084426 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" containerID="cri-o://9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b" gracePeriod=30 Jul 08 22:09:28.336746 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.336721 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:28.514452 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.514414 2563 generic.go:356] "Generic (PLEG): container finished" podID="c69edea7-a78b-49c8-86b4-e991877b8703" containerID="9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b" exitCode=0 Jul 08 22:09:28.514620 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.514472 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" Jul 08 22:09:28.514620 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.514476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" event={"ID":"c69edea7-a78b-49c8-86b4-e991877b8703","Type":"ContainerDied","Data":"9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b"} Jul 08 22:09:28.514703 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.514641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl" event={"ID":"c69edea7-a78b-49c8-86b4-e991877b8703","Type":"ContainerDied","Data":"b520e26d71dd17d321e63d61d67953260fce91aaee89e1f8c1583e44596ae5b4"} Jul 08 22:09:28.514703 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.514676 2563 scope.go:117] "RemoveContainer" containerID="9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b" Jul 08 22:09:28.523093 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.523073 2563 scope.go:117] "RemoveContainer" containerID="2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4" Jul 08 22:09:28.528295 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528271 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-certs\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528395 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528313 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-socket\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528395 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528337 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-data\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528395 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528377 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-credential-socket\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528546 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528409 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-envoy\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528546 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528453 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69edea7-a78b-49c8-86b4-e991877b8703-istio-podinfo\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528546 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528485 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69edea7-a78b-49c8-86b4-e991877b8703-istiod-ca-cert\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528546 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528524 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-istio-token\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528736 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528537 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:09:28.528736 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528570 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsg4m\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-kube-api-access-rsg4m\") pod \"c69edea7-a78b-49c8-86b4-e991877b8703\" (UID: \"c69edea7-a78b-49c8-86b4-e991877b8703\") " Jul 08 22:09:28.528736 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528578 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:09:28.528894 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528842 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.528894 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528841 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:09:28.528894 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528853 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69edea7-a78b-49c8-86b4-e991877b8703-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:09:28.528894 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.528859 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-workload-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.529438 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.529405 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-data" (OuterVolumeSpecName: "istio-data") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:09:28.530638 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.530615 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:09:28.531041 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.531023 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c69edea7-a78b-49c8-86b4-e991877b8703-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 22:09:28.531480 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.531450 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-kube-api-access-rsg4m" (OuterVolumeSpecName: "kube-api-access-rsg4m") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "kube-api-access-rsg4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:09:28.531480 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.531459 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-istio-token" (OuterVolumeSpecName: "istio-token") pod "c69edea7-a78b-49c8-86b4-e991877b8703" (UID: "c69edea7-a78b-49c8-86b4-e991877b8703"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:09:28.532073 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.532052 2563 scope.go:117] "RemoveContainer" containerID="9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b" Jul 08 22:09:28.532379 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:09:28.532360 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b\": container with ID starting with 9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b not found: ID does not exist" containerID="9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b" Jul 08 22:09:28.532455 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.532388 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b"} err="failed to get container status \"9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b\": rpc error: code = NotFound desc = could not find container \"9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b\": container with ID starting with 9e7bbeb0b26eb73ec8daf29485cddd373e95aeb7d600a704112fb03f2627cb6b not found: ID does not exist" Jul 08 22:09:28.532455 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.532406 2563 scope.go:117] "RemoveContainer" containerID="2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4" Jul 08 22:09:28.532636 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:09:28.532621 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4\": container with ID starting with 2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4 not found: ID does not exist" containerID="2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4" Jul 08 22:09:28.532692 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.532639 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4"} err="failed to get container status \"2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4\": rpc error: code = NotFound desc = could not find container \"2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4\": container with ID starting with 2358420745b0db266a72c5396981fd7bdb1c846d08bb2148da4d1c71e48e38e4 not found: ID does not exist" Jul 08 22:09:28.629559 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629528 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c69edea7-a78b-49c8-86b4-e991877b8703-istio-podinfo\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.629559 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629557 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c69edea7-a78b-49c8-86b4-e991877b8703-istiod-ca-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.629559 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629565 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-istio-token\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.629986 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629601 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsg4m\" (UniqueName: \"kubernetes.io/projected/c69edea7-a78b-49c8-86b4-e991877b8703-kube-api-access-rsg4m\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.629986 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629613 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.629986 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629622 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-credential-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.629986 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.629630 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c69edea7-a78b-49c8-86b4-e991877b8703-istio-envoy\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:09:28.837959 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.837918 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl"] Jul 08 22:09:28.848330 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:28.848304 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-648128-openshift-default-5d86999747-7mvwl"] Jul 08 22:09:29.251094 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:29.251046 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" path="/var/lib/kubelet/pods/c69edea7-a78b-49c8-86b4-e991877b8703/volumes" Jul 08 22:09:31.796417 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796381 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj"] Jul 08 22:09:31.796883 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796797 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" Jul 08 22:09:31.796883 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796813 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" Jul 08 22:09:31.796883 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796830 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" Jul 08 22:09:31.796883 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796837 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="efeb02d3-5602-4fe4-b015-eeb0c81aad69" containerName="istio-proxy" Jul 08 22:09:31.796883 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796850 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" Jul 08 22:09:31.796883 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796855 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" Jul 08 22:09:31.797220 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796901 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" Jul 08 22:09:31.797220 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.796909 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69edea7-a78b-49c8-86b4-e991877b8703" containerName="istio-proxy" Jul 08 22:09:31.799937 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.799917 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.802022 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.802002 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ca01c9-openshift-default-dockercfg-dxvd8\"" Jul 08 22:09:31.810888 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.810855 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj"] Jul 08 22:09:31.857628 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-data\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.857814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857636 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-certs\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.857814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-socket\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.857814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946cdc12-69b3-40ef-bdf2-857286c1be2b-istiod-ca-cert\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.857814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-credential-socket\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.857814 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5gfc\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-kube-api-access-d5gfc\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.858026 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-token\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.858026 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-envoy\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.858026 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.857886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-podinfo\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958282 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-credential-socket\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958496 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5gfc\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-kube-api-access-d5gfc\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958496 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-token\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958496 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-envoy\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958496 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-podinfo\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958496 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958447 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-data\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958496 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958476 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-certs\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958874 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958519 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-socket\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958874 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946cdc12-69b3-40ef-bdf2-857286c1be2b-istiod-ca-cert\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958874 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958714 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-credential-socket\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.958874 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958820 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-certs\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.959089 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958969 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-socket\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.959089 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.958985 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-data\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.959310 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.959289 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946cdc12-69b3-40ef-bdf2-857286c1be2b-istiod-ca-cert\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.961062 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.961036 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-podinfo\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.961167 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.961046 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-envoy\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.966110 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.966086 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-token\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:31.966255 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:31.966172 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5gfc\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-kube-api-access-d5gfc\") pod \"e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:32.113462 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.113421 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:32.261311 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.261278 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj"] Jul 08 22:09:32.264605 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:09:32.264582 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946cdc12_69b3_40ef_bdf2_857286c1be2b.slice/crio-bcc5561cff6729a616a3c6b1d1a71f7203170f0d863b523292f2863d2b7b77d2 WatchSource:0}: Error finding container bcc5561cff6729a616a3c6b1d1a71f7203170f0d863b523292f2863d2b7b77d2: Status 404 returned error can't find the container with id bcc5561cff6729a616a3c6b1d1a71f7203170f0d863b523292f2863d2b7b77d2 Jul 08 22:09:32.266640 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.266611 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:32.266756 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.266672 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:32.266756 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.266719 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:32.540724 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.540627 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" event={"ID":"946cdc12-69b3-40ef-bdf2-857286c1be2b","Type":"ContainerStarted","Data":"2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f"} Jul 08 22:09:32.540889 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.540735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" event={"ID":"946cdc12-69b3-40ef-bdf2-857286c1be2b","Type":"ContainerStarted","Data":"bcc5561cff6729a616a3c6b1d1a71f7203170f0d863b523292f2863d2b7b77d2"} Jul 08 22:09:32.560057 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:32.560001 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podStartSLOduration=1.559980571 podStartE2EDuration="1.559980571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:09:32.557918447 +0000 UTC m=+2195.779928628" watchObservedRunningTime="2026-07-08 22:09:32.559980571 +0000 UTC m=+2195.781990773" Jul 08 22:09:33.114624 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:33.114586 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:33.116274 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:33.116247 2563 patch_prober.go:28] interesting pod/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 22:09:33.116409 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:33.116305 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 22:09:34.118734 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:34.118654 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:34.550356 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:34.550269 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:34.551507 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:34.551477 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:48.609675 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:48.609637 2563 generic.go:356] "Generic (PLEG): container finished" podID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerID="2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f" exitCode=0 Jul 08 22:09:48.610172 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:48.609696 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" event={"ID":"946cdc12-69b3-40ef-bdf2-857286c1be2b","Type":"ContainerDied","Data":"2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f"} Jul 08 22:09:48.610172 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:48.610072 2563 scope.go:117] "RemoveContainer" containerID="2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f" Jul 08 22:09:48.611058 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:48.611024 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:48.611195 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:48.611099 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:48.611195 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:48.611161 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 08 22:09:49.550984 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:49.550934 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:49.616207 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:49.616161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" event={"ID":"946cdc12-69b3-40ef-bdf2-857286c1be2b","Type":"ContainerStarted","Data":"fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b"} Jul 08 22:09:50.114860 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:50.114812 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:50.120733 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:50.120185 2563 patch_prober.go:28] interesting pod/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 22:09:50.120733 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:50.120257 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 22:09:51.114707 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:51.114670 2563 patch_prober.go:28] interesting pod/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 22:09:51.115158 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:51.114726 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 22:09:52.114154 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:52.114082 2563 patch_prober.go:28] interesting pod/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 22:09:52.114367 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:52.114177 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 22:09:53.114006 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:53.113966 2563 patch_prober.go:28] interesting pod/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 22:09:53.114523 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:53.114029 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 22:09:54.114774 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:54.114730 2563 patch_prober.go:28] interesting pod/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 08 22:09:54.115312 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:54.114812 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 08 22:09:55.117968 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:55.117935 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:55.118405 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:55.118342 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:09:55.119039 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:09:55.119022 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:12:56.992984 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:12:56.992939 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj"] Jul 08 22:12:56.993463 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:12:56.993217 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" containerID="cri-o://fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b" gracePeriod=30 Jul 08 22:13:02.254793 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.254763 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:13:02.314913 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314817 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-certs\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.314913 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314851 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-token\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.314913 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314899 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5gfc\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-kube-api-access-d5gfc\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314919 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-podinfo\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314937 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-credential-socket\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314961 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946cdc12-69b3-40ef-bdf2-857286c1be2b-istiod-ca-cert\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.314978 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-socket\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315154 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315163 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-data\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315228 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315225 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-envoy\") pod \"946cdc12-69b3-40ef-bdf2-857286c1be2b\" (UID: \"946cdc12-69b3-40ef-bdf2-857286c1be2b\") " Jul 08 22:13:02.315567 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315426 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:13:02.315629 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315555 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946cdc12-69b3-40ef-bdf2-857286c1be2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:13:02.315629 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315611 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-credential-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.315786 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315628 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-certs\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.315979 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.315958 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:13:02.316378 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.316354 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:13:02.317431 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.317406 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:13:02.317530 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.317451 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 22:13:02.317970 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.317950 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:13:02.318037 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.317950 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-kube-api-access-d5gfc" (OuterVolumeSpecName: "kube-api-access-d5gfc") pod "946cdc12-69b3-40ef-bdf2-857286c1be2b" (UID: "946cdc12-69b3-40ef-bdf2-857286c1be2b"). InnerVolumeSpecName "kube-api-access-d5gfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:13:02.416668 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416626 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5gfc\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-kube-api-access-d5gfc\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.416668 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416662 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-podinfo\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.416668 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416673 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946cdc12-69b3-40ef-bdf2-857286c1be2b-istiod-ca-cert\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.416668 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416681 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-workload-socket\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.416945 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416689 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.416945 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416696 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-envoy\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.416945 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.416703 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946cdc12-69b3-40ef-bdf2-857286c1be2b-istio-token\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jul 08 22:13:02.456633 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.456604 2563 generic.go:356] "Generic (PLEG): container finished" podID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerID="fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b" exitCode=0 Jul 08 22:13:02.456783 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.456654 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" Jul 08 22:13:02.456783 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.456693 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" event={"ID":"946cdc12-69b3-40ef-bdf2-857286c1be2b","Type":"ContainerDied","Data":"fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b"} Jul 08 22:13:02.456783 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.456753 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj" event={"ID":"946cdc12-69b3-40ef-bdf2-857286c1be2b","Type":"ContainerDied","Data":"bcc5561cff6729a616a3c6b1d1a71f7203170f0d863b523292f2863d2b7b77d2"} Jul 08 22:13:02.456783 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.456778 2563 scope.go:117] "RemoveContainer" containerID="fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b" Jul 08 22:13:02.466079 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.466061 2563 scope.go:117] "RemoveContainer" containerID="2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f" Jul 08 22:13:02.473987 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.473967 2563 scope.go:117] "RemoveContainer" containerID="fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b" Jul 08 22:13:02.474345 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:13:02.474317 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b\": container with ID starting with fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b not found: ID does not exist" containerID="fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b" Jul 08 22:13:02.474442 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.474355 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b"} err="failed to get container status \"fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b\": rpc error: code = NotFound desc = could not find container \"fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b\": container with ID starting with fe7d0fb0eacf1fb8f232f202bd5512264aa82d944a7c36046f89b3e4e36e9d0b not found: ID does not exist" Jul 08 22:13:02.474442 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.474379 2563 scope.go:117] "RemoveContainer" containerID="2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f" Jul 08 22:13:02.474640 ip-10-0-130-8 kubenswrapper[2563]: E0708 22:13:02.474623 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f\": container with ID starting with 2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f not found: ID does not exist" containerID="2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f" Jul 08 22:13:02.474684 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.474650 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f"} err="failed to get container status \"2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f\": rpc error: code = NotFound desc = could not find container \"2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f\": container with ID starting with 2a3fb2f1315dff1a8058653d70a377e41d37127b9acd2a4d8e7e8fa74014446f not found: ID does not exist" Jul 08 22:13:02.477626 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.477604 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj"] Jul 08 22:13:02.485303 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.482743 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ca01c9-openshift-default-744ff758f9-h8mjj"] Jul 08 22:13:02.527921 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.527896 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7wdjv_f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249/server/0.log" Jul 08 22:13:02.750912 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.750884 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697f9f6747-b56m7_b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d/copy-manifests/0.log" Jul 08 22:13:02.769672 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:02.769631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697f9f6747-b56m7_b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d/manager/0.log" Jul 08 22:13:03.250813 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:03.250776 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" path="/var/lib/kubelet/pods/946cdc12-69b3-40ef-bdf2-857286c1be2b/volumes" Jul 08 22:13:05.382025 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:05.381990 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llcm_aa754c7e-32e9-4b85-a87c-31f2aa76b87c/istio-proxy/0.log" Jul 08 22:13:05.607780 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:05.607746 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dfdc66fb-msdqr_f35861ca-8595-414c-bf05-cbe70c2b6586/kube-auth-proxy/0.log" Jul 08 22:13:06.831892 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:06.831863 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxvk9q_e39faf2e-8d81-4ad1-b9b2-c6a0a4faddd5/main/0.log" Jul 08 22:13:06.935865 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:06.935797 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-44j7c_0a73855c-d84a-4bd0-a403-4ace2d7dc523/main/0.log" Jul 08 22:13:07.043569 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:07.043536 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-8xgkc_0d20aded-57a5-4c9d-801f-767730260df9/main/0.log" Jul 08 22:13:13.714097 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:13.714055 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hqqrf_1a1bcf4f-bebf-4346-8da2-5dd2f80d90b9/global-pull-secret-syncer/0.log" Jul 08 22:13:13.915544 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:13.915509 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zvbhb_6e23b995-db34-4684-8653-ecd69281486a/konnectivity-agent/0.log" Jul 08 22:13:13.934975 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:13.934947 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-8.ec2.internal_ff34105d0dcd17ef37ae6f670566787e/haproxy/0.log" Jul 08 22:13:21.021418 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.021327 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-jltb7_ef134186-603f-4d09-9547-7edc81b22799/kube-state-metrics/0.log" Jul 08 22:13:21.053563 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.053530 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-jltb7_ef134186-603f-4d09-9547-7edc81b22799/kube-rbac-proxy-main/0.log" Jul 08 22:13:21.075931 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.075900 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-jltb7_ef134186-603f-4d09-9547-7edc81b22799/kube-rbac-proxy-self/0.log" Jul 08 22:13:21.154351 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.154325 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-6l9lq_fa65a4d2-0fcb-424b-810d-6f4fe3c88302/monitoring-plugin/0.log" Jul 08 22:13:21.189692 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.189659 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5vpzn_989a4096-a50c-4828-a7c3-9e3845df3a4b/node-exporter/0.log" Jul 08 22:13:21.221855 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.221815 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5vpzn_989a4096-a50c-4828-a7c3-9e3845df3a4b/kube-rbac-proxy/0.log" Jul 08 22:13:21.249302 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:21.249277 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5vpzn_989a4096-a50c-4828-a7c3-9e3845df3a4b/init-textfile/0.log" Jul 08 22:13:24.389298 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389262 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j"] Jul 08 22:13:24.389787 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389576 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" Jul 08 22:13:24.389787 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389587 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" Jul 08 22:13:24.389787 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389603 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" Jul 08 22:13:24.389787 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389609 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" Jul 08 22:13:24.389787 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389669 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" Jul 08 22:13:24.389787 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.389677 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="946cdc12-69b3-40ef-bdf2-857286c1be2b" containerName="istio-proxy" Jul 08 22:13:24.393946 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.393914 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.396185 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.396161 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rf8nv\"/\"openshift-service-ca.crt\"" Jul 08 22:13:24.396933 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.396912 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rf8nv\"/\"kube-root-ca.crt\"" Jul 08 22:13:24.397050 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.396917 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rf8nv\"/\"default-dockercfg-zdh2k\"" Jul 08 22:13:24.402586 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.402560 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j"] Jul 08 22:13:24.482321 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.482292 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f58fdc776-q76d9_27cdaf17-d31c-4760-9328-20996b5f04bf/console/0.log" Jul 08 22:13:24.519797 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.519760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-sys\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.520003 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.519822 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-proc\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.520003 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.519917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-podres\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.520003 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.519963 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x5kq\" (UniqueName: \"kubernetes.io/projected/b473fcde-416f-4650-b8c3-2e7d80da4b64-kube-api-access-2x5kq\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.520225 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.520044 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-lib-modules\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.520791 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.520766 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-kndkl_a29557d8-d381-43f7-8096-c9fc32ce18b3/download-server/0.log" Jul 08 22:13:24.620497 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x5kq\" (UniqueName: \"kubernetes.io/projected/b473fcde-416f-4650-b8c3-2e7d80da4b64-kube-api-access-2x5kq\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620689 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620525 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-lib-modules\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620689 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620554 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-sys\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620689 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-proc\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620689 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-podres\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620689 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620662 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-lib-modules\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620689 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620682 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-proc\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620916 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620670 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-sys\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.620916 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.620737 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b473fcde-416f-4650-b8c3-2e7d80da4b64-podres\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.628782 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.628743 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x5kq\" (UniqueName: \"kubernetes.io/projected/b473fcde-416f-4650-b8c3-2e7d80da4b64-kube-api-access-2x5kq\") pod \"perf-node-gather-daemonset-6pz5j\" (UID: \"b473fcde-416f-4650-b8c3-2e7d80da4b64\") " pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.704989 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.704890 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:24.836424 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:24.836366 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j"] Jul 08 22:13:24.838627 ip-10-0-130-8 kubenswrapper[2563]: W0708 22:13:24.838597 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb473fcde_416f_4650_b8c3_2e7d80da4b64.slice/crio-3fca16b769d2ec9b7e5a652a290dbff0ed0b06b30848792214d60afa26cf07c5 WatchSource:0}: Error finding container 3fca16b769d2ec9b7e5a652a290dbff0ed0b06b30848792214d60afa26cf07c5: Status 404 returned error can't find the container with id 3fca16b769d2ec9b7e5a652a290dbff0ed0b06b30848792214d60afa26cf07c5 Jul 08 22:13:25.556924 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:25.556884 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" event={"ID":"b473fcde-416f-4650-b8c3-2e7d80da4b64","Type":"ContainerStarted","Data":"fceb266385fb7768492c7bc2457a807ad6ef69d7ba644eeb749fef1e77ee3a6b"} Jul 08 22:13:25.557365 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:25.556947 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:25.557365 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:25.556968 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" event={"ID":"b473fcde-416f-4650-b8c3-2e7d80da4b64","Type":"ContainerStarted","Data":"3fca16b769d2ec9b7e5a652a290dbff0ed0b06b30848792214d60afa26cf07c5"} Jul 08 22:13:25.571455 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:25.571409 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" podStartSLOduration=1.571391336 podStartE2EDuration="1.571391336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:13:25.570799012 +0000 UTC m=+2428.792809214" watchObservedRunningTime="2026-07-08 22:13:25.571391336 +0000 UTC m=+2428.793401536" Jul 08 22:13:26.176160 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:26.176110 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-86qgs_a694888b-76c7-4bc4-93bf-5ad59b2d7bdc/dns/0.log" Jul 08 22:13:26.195908 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:26.195878 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-86qgs_a694888b-76c7-4bc4-93bf-5ad59b2d7bdc/kube-rbac-proxy/0.log" Jul 08 22:13:26.285862 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:26.285826 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fgr7d_ed16c060-6133-43f3-9eb5-5bbc58e352f0/dns-node-resolver/0.log" Jul 08 22:13:26.854095 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:26.854063 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-799698dd98-fc2q4_71f3f18a-f390-42ca-9357-14f8baf01c3d/registry/0.log" Jul 08 22:13:26.877592 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:26.877557 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ddhjc_e6304ba3-31c3-46c9-aa0a-0d9397cbdd27/node-ca/0.log" Jul 08 22:13:27.837985 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:27.837949 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llcm_aa754c7e-32e9-4b85-a87c-31f2aa76b87c/istio-proxy/0.log" Jul 08 22:13:28.040411 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:28.040374 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dfdc66fb-msdqr_f35861ca-8595-414c-bf05-cbe70c2b6586/kube-auth-proxy/0.log" Jul 08 22:13:29.015101 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:29.015071 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d8qct_b2922392-a0bd-416b-8189-4c84842e19e0/serve-healthcheck-canary/0.log" Jul 08 22:13:29.516208 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:29.516177 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9tgvh_b70cc2a2-566c-4164-82bb-7b264d24c002/kube-rbac-proxy/0.log" Jul 08 22:13:29.536258 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:29.536218 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9tgvh_b70cc2a2-566c-4164-82bb-7b264d24c002/exporter/0.log" Jul 08 22:13:29.557312 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:29.557281 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9tgvh_b70cc2a2-566c-4164-82bb-7b264d24c002/extractor/0.log" Jul 08 22:13:31.571074 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:31.571045 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rf8nv/perf-node-gather-daemonset-6pz5j" Jul 08 22:13:32.090263 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:32.090236 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7wdjv_f4dd7d70-9c8d-40ca-b6df-5f8ab8a28249/server/0.log" Jul 08 22:13:32.255754 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:32.255709 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697f9f6747-b56m7_b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d/manager/0.log" Jul 08 22:13:32.271307 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:32.271279 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697f9f6747-b56m7_b22585eb-ee29-40ce-a8fb-9c2c4cd33c5d/copy-manifests/0.log" Jul 08 22:13:33.650431 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:33.650400 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5947469944-fx6bz_d426944a-6e3b-4177-a1b5-64b6626c28ef/manager/0.log" Jul 08 22:13:34.240283 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:34.240239 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llcm_aa754c7e-32e9-4b85-a87c-31f2aa76b87c/istio-proxy/0.log" Jul 08 22:13:34.413635 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:34.413605 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dfdc66fb-msdqr_f35861ca-8595-414c-bf05-cbe70c2b6586/kube-auth-proxy/0.log" Jul 08 22:13:35.809028 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:35.808993 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llcm_aa754c7e-32e9-4b85-a87c-31f2aa76b87c/istio-proxy/0.log" Jul 08 22:13:35.983330 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:35.983295 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dfdc66fb-msdqr_f35861ca-8595-414c-bf05-cbe70c2b6586/kube-auth-proxy/0.log" Jul 08 22:13:40.254014 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.253985 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4f6wl_2f67521e-4089-4ab3-8923-40ad7bc1d024/kube-multus/0.log" Jul 08 22:13:40.277865 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.277834 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/kube-multus-additional-cni-plugins/0.log" Jul 08 22:13:40.298277 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.298251 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/egress-router-binary-copy/0.log" Jul 08 22:13:40.318904 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.318877 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/cni-plugins/0.log" Jul 08 22:13:40.339265 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.339226 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/bond-cni-plugin/0.log" Jul 08 22:13:40.392991 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.392958 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/routeoverride-cni/0.log" Jul 08 22:13:40.440972 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.440889 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/whereabouts-cni-bincopy/0.log" Jul 08 22:13:40.496486 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.496451 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qh4v8_6bfb4fef-2218-465f-95a6-8515c5c7c77a/whereabouts-cni/0.log" Jul 08 22:13:40.987180 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:40.987119 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zqhjt_95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c/network-metrics-daemon/0.log" Jul 08 22:13:41.005458 ip-10-0-130-8 kubenswrapper[2563]: I0708 22:13:41.005426 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zqhjt_95e42ff9-cf62-4ff2-96e5-2dabbdac2e7c/kube-rbac-proxy/0.log"