Jul 08 06:52:39.900059 ip-10-0-138-98 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 06:52:39.900072 ip-10-0-138-98 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 06:52:39.900081 ip-10-0-138-98 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 06:52:39.900385 ip-10-0-138-98 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 06:52:50.149588 ip-10-0-138-98 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 06:52:50.149606 ip-10-0-138-98 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 86ebe4a9fb944bccaae0195ff3360753 -- Jul 08 06:54:47.174232 ip-10-0-138-98 systemd[1]: Starting Kubernetes Kubelet... Jul 08 06:54:47.558841 ip-10-0-138-98 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 06:54:47.558841 ip-10-0-138-98 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 06:54:47.558841 ip-10-0-138-98 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 06:54:47.558841 ip-10-0-138-98 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 06:54:47.558841 ip-10-0-138-98 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 06:54:47.560291 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.560204 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 06:54:47.563464 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563448 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 06:54:47.563464 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563464 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563469 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563474 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563478 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563482 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563485 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563487 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563490 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563493 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563495 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563498 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563501 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563503 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563506 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563509 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563511 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563514 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563517 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563523 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 06:54:47.563525 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563526 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563528 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563532 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563535 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563537 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563540 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563543 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563546 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563548 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563551 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563553 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563556 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563558 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563561 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563564 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563567 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563570 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563572 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563575 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563578 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 06:54:47.563976 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563582 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563585 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563588 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563590 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563593 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563596 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563599 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563601 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563604 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563606 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563609 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563611 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563613 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563616 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563619 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563622 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563624 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563627 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563629 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563631 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 06:54:47.564487 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563636 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563639 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563642 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563645 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563649 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563651 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563654 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563657 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563660 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563662 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563665 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563667 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563670 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563672 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563675 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563677 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563680 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563682 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563685 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563688 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 06:54:47.564967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563691 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563693 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563696 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563699 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563701 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563704 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563708 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563711 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563713 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563715 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563718 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563721 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563723 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563726 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563728 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563731 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563733 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563736 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.563738 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 06:54:47.565451 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564433 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564440 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564443 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564446 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564448 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564451 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564454 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564457 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564460 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564462 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564465 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564467 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564471 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564474 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564476 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564479 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564482 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564484 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564487 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564490 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 06:54:47.565959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564493 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564496 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564500 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564504 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564507 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564509 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564512 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564514 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564517 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564519 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564522 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564525 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564527 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564530 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564532 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564535 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564537 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564540 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564542 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 06:54:47.566467 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564545 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564547 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564550 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564553 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564556 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564559 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564562 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564565 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564567 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564570 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564572 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564575 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564579 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564581 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564583 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564586 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564588 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564591 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564593 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564595 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 06:54:47.566917 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564600 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564603 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564606 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564610 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564612 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564615 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564618 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564620 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564623 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564626 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564629 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564631 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564635 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564637 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564640 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564643 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564645 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564648 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564650 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564654 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 06:54:47.567404 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564656 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564658 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564661 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564663 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564666 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564668 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564671 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564674 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564676 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564679 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564681 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564684 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564686 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564689 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564691 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564694 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564698 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564701 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564704 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 06:54:47.567888 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.564706 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565134 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565140 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565144 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565147 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565149 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565166 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565170 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565173 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565176 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565178 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565181 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565184 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565186 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565189 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565191 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565194 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565197 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565200 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565202 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 06:54:47.568370 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565206 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565209 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565212 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565215 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565218 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565221 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565224 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565227 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565229 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565232 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565235 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565238 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565241 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565244 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565247 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565250 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565253 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565255 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565258 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565261 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 06:54:47.568852 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565264 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565266 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565269 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565271 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565275 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565279 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565282 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565285 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565287 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565290 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565293 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565295 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565298 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565300 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565303 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565306 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565309 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565311 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565314 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565316 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 06:54:47.569341 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565319 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565321 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565323 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565326 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565329 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565331 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565334 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565337 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565339 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565342 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565345 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565347 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565351 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565354 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565357 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565360 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565363 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565365 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565369 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 06:54:47.569832 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565371 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565374 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565377 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565379 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565382 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565384 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565387 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565391 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565394 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565396 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565399 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565401 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565404 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565406 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565408 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565411 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565414 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565416 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565419 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565421 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 06:54:47.570304 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.565424 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565498 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565505 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565512 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565519 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565523 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565526 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565531 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565535 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565538 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565541 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565545 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565548 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565552 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565555 2568 flags.go:64] FLAG: --cgroup-root="" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565557 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565560 2568 flags.go:64] FLAG: --client-ca-file="" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565563 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565566 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565573 2568 flags.go:64] FLAG: --cluster-domain="" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565575 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565578 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565581 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565584 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 06:54:47.570808 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565588 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565592 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565595 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565598 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565601 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565604 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565607 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565613 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565616 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565620 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565623 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565626 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565629 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565632 2568 flags.go:64] FLAG: --enable-server="true" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565635 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565639 2568 flags.go:64] FLAG: --event-burst="100" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565642 2568 flags.go:64] FLAG: --event-qps="50" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565645 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565647 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565650 2568 flags.go:64] FLAG: --eviction-hard="" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565654 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565658 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565661 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565663 2568 flags.go:64] FLAG: --eviction-soft="" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565666 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 06:54:47.571428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565669 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565672 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565676 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565679 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565681 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565684 2568 flags.go:64] FLAG: --feature-gates="" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565688 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565691 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565694 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565697 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565699 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565702 2568 flags.go:64] FLAG: --help="false" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565705 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565708 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565713 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565716 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565719 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565722 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565725 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565728 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565730 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565733 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565736 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565739 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 06:54:47.572026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565742 2568 flags.go:64] FLAG: --kube-reserved="" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565745 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565747 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565751 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565753 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565757 2568 flags.go:64] FLAG: --lock-file="" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565759 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565762 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565765 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565770 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565775 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565785 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565789 2568 flags.go:64] FLAG: --logging-format="text" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565792 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565795 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565798 2568 flags.go:64] FLAG: --manifest-url="" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565801 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565805 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565808 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565812 2568 flags.go:64] FLAG: --max-pods="110" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565815 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565819 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565824 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565827 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565830 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 06:54:47.572648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565833 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565836 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565843 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565846 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565849 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565852 2568 flags.go:64] FLAG: --pod-cidr="" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565855 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565861 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565863 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565866 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565869 2568 flags.go:64] FLAG: --port="10250" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565872 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565875 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fca093ff5623f5d1" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565879 2568 flags.go:64] FLAG: --qos-reserved="" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565882 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565885 2568 flags.go:64] FLAG: --register-node="true" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565888 2568 flags.go:64] FLAG: --register-with-taints="" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565893 2568 flags.go:64] FLAG: --registry-burst="10" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565896 2568 flags.go:64] FLAG: --registry-qps="5" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565899 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565901 2568 flags.go:64] FLAG: --reserved-memory="" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565905 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565908 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565911 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565914 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 06:54:47.573268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565916 2568 flags.go:64] FLAG: --runonce="false" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565919 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565922 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565925 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565928 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565932 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565935 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565938 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565942 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565944 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565948 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565951 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565953 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565956 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565959 2568 flags.go:64] FLAG: --system-cgroups="" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565962 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565967 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565970 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565973 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565978 2568 flags.go:64] FLAG: --tls-min-version="" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565981 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565984 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565987 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565989 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565993 2568 flags.go:64] FLAG: --v="2" Jul 08 06:54:47.573885 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.565997 2568 flags.go:64] FLAG: --version="false" Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.566001 2568 flags.go:64] FLAG: --vmodule="" Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.566004 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.566008 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566120 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566123 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566127 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566130 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566133 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566137 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566139 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566142 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566147 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566149 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566165 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566168 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566171 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566173 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566176 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566178 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566181 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 06:54:47.574959 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566183 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566186 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566188 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566191 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566194 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566196 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566199 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566201 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566204 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566207 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566211 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566213 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566216 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566218 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566221 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566223 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566226 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566228 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566231 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 06:54:47.575516 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566233 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566236 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566238 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566241 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566245 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566247 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566252 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566255 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566258 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566261 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566264 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566266 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566269 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566271 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566274 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566276 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566280 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566282 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566285 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 06:54:47.575984 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566287 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566290 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566292 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566295 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566299 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566301 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566304 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566306 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566309 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566311 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566314 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566318 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566321 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566324 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566327 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566330 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566333 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566336 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566339 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566341 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 06:54:47.576470 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566344 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566347 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566349 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566351 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566354 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566356 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566359 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566361 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566364 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566366 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566369 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566371 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566374 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566378 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566381 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566384 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566389 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566391 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566394 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566396 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 06:54:47.576967 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566399 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566401 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566404 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.566406 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.567352 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.575804 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.575820 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.575841 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.575847 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575905 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575910 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575914 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575917 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575920 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575923 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 06:54:47.577476 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575926 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575929 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575932 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575935 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575938 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575940 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575943 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575946 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575949 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575951 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575954 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575956 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575959 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575962 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575964 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575967 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575974 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575977 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575980 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575983 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 06:54:47.577857 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575985 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575988 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575991 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575994 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.575997 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576000 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576002 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576005 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576007 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576010 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576013 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576015 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576018 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576021 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576023 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576026 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576028 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576031 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576033 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576036 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 06:54:47.578356 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576039 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576041 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576044 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576047 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576051 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576055 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576058 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576061 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576064 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576067 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576069 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576072 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576075 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576078 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576080 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576083 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576086 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576088 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576091 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 06:54:47.578845 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576094 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576096 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576098 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576101 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576103 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576106 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576109 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576111 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576114 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576118 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576122 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576124 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576127 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576130 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576133 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576137 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576141 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576143 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576146 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 06:54:47.579337 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576149 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576151 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576170 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576173 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576176 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576178 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576181 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576183 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576186 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576190 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576192 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576195 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576198 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576200 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 06:54:47.579804 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576203 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.576208 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576351 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576357 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576360 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576363 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576366 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576369 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576372 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576375 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576378 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576380 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576383 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576386 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576389 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 06:54:47.580182 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576391 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576394 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576396 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576399 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576402 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576404 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576407 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576409 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576412 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576414 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576416 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576419 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576422 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576425 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576427 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576430 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576432 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576435 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576438 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 06:54:47.580557 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576440 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576443 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576445 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576447 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576450 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576452 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576455 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576457 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576460 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576462 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576464 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576468 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576471 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576474 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576476 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576479 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576482 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576485 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576488 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576490 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 06:54:47.581016 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576493 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576495 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576498 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576500 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576504 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576508 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576511 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576513 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576516 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576519 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576521 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576525 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576528 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576530 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576533 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576535 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576538 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576540 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576543 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 06:54:47.581621 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576547 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576550 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576553 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576556 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576560 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576563 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576566 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576569 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576572 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576575 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576577 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576580 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576582 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576585 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576587 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576590 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576592 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576595 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576598 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576600 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 06:54:47.582095 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576603 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576606 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576608 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576611 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576613 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576616 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576618 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:47.576621 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.576625 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.576736 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.580246 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.581545 2568 server.go:1019] "Starting client certificate rotation" Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.581648 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 06:54:47.582586 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.581686 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 06:54:47.583588 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.583574 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 06:54:47.584784 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.584766 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 06:54:47.596456 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.596441 2568 log.go:25] "Validated CRI v1 runtime API" Jul 08 06:54:47.598686 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.598668 2568 log.go:25] "Validated CRI v1 image API" Jul 08 06:54:47.599581 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.599564 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 06:54:47.603729 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.603702 2568 fs.go:135] Filesystem UUIDs: map[3f0c0127-5722-4329-bb3b-ac27aa5fe0a1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f43c2108-e0d7-43ba-957c-576e582ab3ca:/dev/nvme0n1p4] Jul 08 06:54:47.603803 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.603726 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 06:54:47.609620 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.609600 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 06:54:47.609988 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.609867 2568 manager.go:216] Machine: {Timestamp:2026-07-08 06:54:47.607720961 +0000 UTC m=+0.321369792 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3087639 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bcf13b18db719374c2235f7ccde51 SystemUUID:ec2bcf13-b18d-b719-374c-2235f7ccde51 BootID:86ebe4a9-fb94-4bcc-aae0-195ff3360753 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:bc:9b:dd:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:bc:9b:dd:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:71:0d:da:e6:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 06:54:47.609988 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.609987 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 06:54:47.610119 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.610107 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 06:54:47.614411 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.614197 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 06:54:47.614602 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.614416 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-98.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 06:54:47.615202 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.615191 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 06:54:47.615250 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.615205 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 06:54:47.615250 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.615214 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 06:54:47.615969 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.615957 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 06:54:47.617265 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.617254 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 06:54:47.621055 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621042 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 08 06:54:47.621171 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621161 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 06:54:47.621204 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621193 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 08 06:54:47.621255 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621204 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 06:54:47.621255 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621223 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 06:54:47.621255 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621246 2568 kubelet.go:398] "Adding apiserver pod source" Jul 08 06:54:47.621255 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621256 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 06:54:47.621709 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621696 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 06:54:47.621772 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.621715 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 06:54:47.623953 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.623934 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 06:54:47.625605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.625587 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 06:54:47.625698 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.625628 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 06:54:47.626782 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626768 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626789 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626799 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626808 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626821 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626830 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626842 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 06:54:47.626854 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626851 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 06:54:47.627087 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626862 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 06:54:47.627087 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626871 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 06:54:47.627087 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626885 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 06:54:47.627087 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.626894 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 06:54:47.628285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.628272 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 06:54:47.628353 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.628287 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 06:54:47.632273 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.632255 2568 server.go:1268] "Started kubelet" Jul 08 06:54:47.632371 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.632341 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 06:54:47.632486 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.632411 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 06:54:47.632540 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.632514 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 06:54:47.632692 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.632642 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-98.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 06:54:47.632754 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.632719 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 06:54:47.632754 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.632738 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 06:54:47.633783 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.633760 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 06:54:47.634452 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.634435 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 08 06:54:47.634526 ip-10-0-138-98 systemd[1]: Started Kubernetes Kubelet. Jul 08 06:54:47.638890 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.637785 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-98.ec2.internal.18c03e0f3e7810fa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-98.ec2.internal,UID:ip-10-0-138-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-98.ec2.internal,},FirstTimestamp:2026-07-08 06:54:47.632204026 +0000 UTC m=+0.345852856,LastTimestamp:2026-07-08 06:54:47.632204026 +0000 UTC m=+0.345852856,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-98.ec2.internal,}" Jul 08 06:54:47.639052 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639028 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 06:54:47.639222 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639058 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 06:54:47.639292 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.639263 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:47.639594 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.639553 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 06:54:47.639791 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639770 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 06:54:47.639791 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639790 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 06:54:47.639924 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639769 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 06:54:47.639924 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639838 2568 factory.go:55] Registering systemd factory Jul 08 06:54:47.639924 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639854 2568 factory.go:223] Registration of the systemd container factory successfully Jul 08 06:54:47.639924 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639893 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 08 06:54:47.639924 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.639903 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 08 06:54:47.640196 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.640083 2568 factory.go:153] Registering CRI-O factory Jul 08 06:54:47.640196 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.640096 2568 factory.go:223] Registration of the crio container factory successfully Jul 08 06:54:47.640335 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.640197 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 06:54:47.640335 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.640223 2568 factory.go:103] Registering Raw factory Jul 08 06:54:47.640335 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.640236 2568 manager.go:1196] Started watching for new ooms in manager Jul 08 06:54:47.640653 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.640639 2568 manager.go:349] Starting recovery of all containers Jul 08 06:54:47.642119 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.642097 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8dhg5" Jul 08 06:54:47.648162 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.648128 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 06:54:47.648294 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.648162 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 06:54:47.651413 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.651389 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8dhg5" Jul 08 06:54:47.655035 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.655018 2568 manager.go:354] Recovery completed Jul 08 06:54:47.656480 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.656463 2568 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 08 06:54:47.659569 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.659555 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.661483 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.661466 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.661558 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.661503 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.661558 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.661519 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.662467 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.662446 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 06:54:47.662467 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.662460 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 06:54:47.662604 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.662488 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 08 06:54:47.662799 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.662785 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.664033 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.664014 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.664105 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.664041 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.664105 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.664051 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.664212 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.664139 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-98.ec2.internal.18c03e0f4036dc0c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-98.ec2.internal,UID:ip-10-0-138-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-98.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-98.ec2.internal,},FirstTimestamp:2026-07-08 06:54:47.661485068 +0000 UTC m=+0.375133903,LastTimestamp:2026-07-08 06:54:47.661485068 +0000 UTC m=+0.375133903,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-98.ec2.internal,}" Jul 08 06:54:47.664627 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.664613 2568 policy_none.go:49] "None policy: Start" Jul 08 06:54:47.664679 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.664630 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 06:54:47.664679 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.664644 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 06:54:47.666664 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.666652 2568 policy_none.go:47] "Start" Jul 08 06:54:47.715852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.715829 2568 manager.go:337] "Starting Device Plugin manager" Jul 08 06:54:47.715962 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.715923 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 06:54:47.715962 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.715942 2568 server.go:87] "Starting device plugin registration server" Jul 08 06:54:47.716242 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.716214 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 06:54:47.716343 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.716253 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 06:54:47.716405 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.716380 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 06:54:47.716516 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.716483 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 06:54:47.716516 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.716500 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 06:54:47.717113 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.717093 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 06:54:47.717191 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.717155 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:47.816811 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.816735 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.817349 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.817995 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.818043 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.818060 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.818098 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.818901 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.818941 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.818964 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 06:54:47.819080 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.818997 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 06:54:47.820932 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.820915 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 06:54:47.826947 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.826925 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.827033 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.826951 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-98.ec2.internal\": node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:47.877893 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.877865 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:47.921901 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.921855 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal"] Jul 08 06:54:47.922029 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.921987 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.923052 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.923036 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.923124 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.923066 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.923124 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.923077 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.925633 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.925619 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.925819 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.925804 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.925860 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.925842 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.926661 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.926643 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.926661 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.926654 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.926779 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.926673 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.926779 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.926697 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.926779 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.926710 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.926779 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.926699 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.929099 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.929085 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.929191 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.929112 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 06:54:47.929909 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.929890 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientMemory" Jul 08 06:54:47.929988 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.929926 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 06:54:47.929988 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.929940 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeHasSufficientPID" Jul 08 06:54:47.941292 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.941267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44133c4c266a4c88a4c5bdd8cf3600bf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal\" (UID: \"44133c4c266a4c88a4c5bdd8cf3600bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.941366 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.941301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cafb6815aa15ff96ffbd1f08c40a3af8-config\") pod \"kube-apiserver-proxy-ip-10-0-138-98.ec2.internal\" (UID: \"cafb6815aa15ff96ffbd1f08c40a3af8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.941366 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:47.941338 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44133c4c266a4c88a4c5bdd8cf3600bf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal\" (UID: \"44133c4c266a4c88a4c5bdd8cf3600bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.946825 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.946808 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-98.ec2.internal\" not found" node="ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.950669 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.950652 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-98.ec2.internal\" not found" node="ip-10-0-138-98.ec2.internal" Jul 08 06:54:47.978358 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:47.978335 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.042508 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.042479 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44133c4c266a4c88a4c5bdd8cf3600bf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal\" (UID: \"44133c4c266a4c88a4c5bdd8cf3600bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.042617 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.042557 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44133c4c266a4c88a4c5bdd8cf3600bf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal\" (UID: \"44133c4c266a4c88a4c5bdd8cf3600bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.042617 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.042583 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44133c4c266a4c88a4c5bdd8cf3600bf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal\" (UID: \"44133c4c266a4c88a4c5bdd8cf3600bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.042617 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.042588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44133c4c266a4c88a4c5bdd8cf3600bf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal\" (UID: \"44133c4c266a4c88a4c5bdd8cf3600bf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.042714 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.042635 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cafb6815aa15ff96ffbd1f08c40a3af8-config\") pod \"kube-apiserver-proxy-ip-10-0-138-98.ec2.internal\" (UID: \"cafb6815aa15ff96ffbd1f08c40a3af8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.042714 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.042670 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cafb6815aa15ff96ffbd1f08c40a3af8-config\") pod \"kube-apiserver-proxy-ip-10-0-138-98.ec2.internal\" (UID: \"cafb6815aa15ff96ffbd1f08c40a3af8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.079788 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.079708 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.182210 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.182146 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.251735 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.251711 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.254047 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.254027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" Jul 08 06:54:48.283405 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.283375 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.386776 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.386679 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.487719 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.487680 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.583390 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.583361 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 06:54:48.584026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.583533 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 06:54:48.587873 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.587847 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.623311 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.623281 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 06:54:48.656265 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.656174 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 06:49:48 +0000 UTC" deadline="2027-12-15 17:17:08.899351196 +0000 UTC" Jul 08 06:54:48.656265 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.656208 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12610h22m20.243146341s" Jul 08 06:54:48.688390 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.688359 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.788524 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.788483 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.851458 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:48.851418 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcafb6815aa15ff96ffbd1f08c40a3af8.slice/crio-9776d396da64a357a2e914f8ee986f27ed2b22bfb01596dff07ef8a4204142e5 WatchSource:0}: Error finding container 9776d396da64a357a2e914f8ee986f27ed2b22bfb01596dff07ef8a4204142e5: Status 404 returned error can't find the container with id 9776d396da64a357a2e914f8ee986f27ed2b22bfb01596dff07ef8a4204142e5 Jul 08 06:54:48.852149 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:48.852128 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44133c4c266a4c88a4c5bdd8cf3600bf.slice/crio-461a63ad1106e3c1b2c787f364ad399b23d0402e947611754bdc08d1b2863ac2 WatchSource:0}: Error finding container 461a63ad1106e3c1b2c787f364ad399b23d0402e947611754bdc08d1b2863ac2: Status 404 returned error can't find the container with id 461a63ad1106e3c1b2c787f364ad399b23d0402e947611754bdc08d1b2863ac2 Jul 08 06:54:48.858133 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.858114 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 06:54:48.889149 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:48.889125 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-98.ec2.internal\" not found" Jul 08 06:54:48.963056 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:48.963027 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 06:54:49.002144 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.001331 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 06:54:49.039988 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.039950 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" Jul 08 06:54:49.062949 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.062925 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 06:54:49.063803 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.063789 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" Jul 08 06:54:49.077112 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.077091 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 06:54:49.622328 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.622290 2568 apiserver.go:52] "Watching apiserver" Jul 08 06:54:49.631476 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.631358 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 06:54:49.631848 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.631809 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-q57p9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal","openshift-multus/multus-47vfx","openshift-multus/multus-additional-cni-plugins-69mr4","openshift-network-diagnostics/network-check-target-r7zbh","kube-system/global-pull-secret-syncer-5mkr5","kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz","openshift-cluster-node-tuning-operator/tuned-wp9fw","openshift-dns/node-resolver-zqjwx","openshift-multus/network-metrics-daemon-kmdj4","openshift-network-operator/iptables-alerter-5t9nd","openshift-ovn-kubernetes/ovnkube-node-4cknp","kube-system/konnectivity-agent-qcf6t"] Jul 08 06:54:49.634597 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.634574 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.636918 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.636895 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.637479 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.637445 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kvgbq\"" Jul 08 06:54:49.637794 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.637723 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.637794 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.637749 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 06:54:49.637959 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.637729 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.639318 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.639112 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mxgmc\"" Jul 08 06:54:49.639318 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.639217 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 06:54:49.640029 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.639568 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 06:54:49.640029 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.639650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.640029 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.639947 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 06:54:49.642852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.642074 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.642852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.642243 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 06:54:49.642852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.642288 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 06:54:49.642852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.642493 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.642852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.642536 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.642852 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.642706 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tsx7q\"" Jul 08 06:54:49.644464 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.644301 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 06:54:49.644791 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.644632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:49.644791 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.644713 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:54:49.645143 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.644978 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-brpzq\"" Jul 08 06:54:49.645143 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.645022 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 06:54:49.646840 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.646741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.646840 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.646809 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:54:49.649972 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.649951 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.652494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.652135 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 06:54:49.652494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.652408 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.652638 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.652615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.652767 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.652742 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.653344 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653316 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqv5j\" (UniqueName: \"kubernetes.io/projected/51b3fa95-4da3-454c-8026-2d6e9469fae5-kube-api-access-kqv5j\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.653446 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-cnibin\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653446 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653390 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b62rw\"" Jul 08 06:54:49.653446 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-cni-binary-copy\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653472 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-kubelet\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653502 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-os-release\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.653605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51b3fa95-4da3-454c-8026-2d6e9469fae5-host-slash\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.653605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-socket-dir-parent\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-system-cni-dir\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-cnibin\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653636 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-cni-binary-copy\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653681 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da0da73e-1565-4cd1-a906-37b8316b63e6-konnectivity-ca\") pod \"konnectivity-agent-qcf6t\" (UID: \"da0da73e-1565-4cd1-a906-37b8316b63e6\") " pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653754 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-os-release\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653779 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-netns\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-cni-multus\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.653858 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-hostroot\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6llz4\" (UniqueName: \"kubernetes.io/projected/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-kube-api-access-6llz4\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51b3fa95-4da3-454c-8026-2d6e9469fae5-iptables-alerter-script\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653931 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-conf-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-daemon-config\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.653981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-multus-certs\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da0da73e-1565-4cd1-a906-37b8316b63e6-agent-certs\") pod \"konnectivity-agent-qcf6t\" (UID: \"da0da73e-1565-4cd1-a906-37b8316b63e6\") " pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-etc-kubernetes\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654085 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654112 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4v7x\" (UniqueName: \"kubernetes.io/projected/13995368-b3df-40c6-a914-591ec9da1ded-kube-api-access-n4v7x\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654136 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-cni-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654160 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-cni-bin\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654895 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-system-cni-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.654895 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.654351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-k8s-cni-cncf-io\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.655128 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.655015 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.655128 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.655091 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 06:54:49.655260 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.655218 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zxjl2\"" Jul 08 06:54:49.655708 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.655541 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.655708 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.655585 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.659768 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.658819 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 06:54:49.661106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.660565 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fddqr\"" Jul 08 06:54:49.661106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.660682 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.661106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.660689 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.662728 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.662199 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.664679 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.664655 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.664784 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.664678 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 06:54:49.664784 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.664690 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.664897 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.664840 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 06:54:49.664897 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.664844 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 06:54:49.664897 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.664868 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tfn8z\"" Jul 08 06:54:49.665326 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.665305 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:49.665437 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.665391 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:54:49.665977 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.665956 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 06:54:49.667797 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.667773 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.670200 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.669972 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 06:54:49.670200 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.670043 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 06:54:49.670200 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.670060 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lllc2\"" Jul 08 06:54:49.677933 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.677911 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w9v2h" Jul 08 06:54:49.684095 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.684073 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w9v2h" Jul 08 06:54:49.741593 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.741556 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 06:54:49.755637 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da0da73e-1565-4cd1-a906-37b8316b63e6-konnectivity-ca\") pod \"konnectivity-agent-qcf6t\" (UID: \"da0da73e-1565-4cd1-a906-37b8316b63e6\") " pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.755800 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-os-release\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.755800 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-hostroot\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.755800 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6llz4\" (UniqueName: \"kubernetes.io/projected/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-kube-api-access-6llz4\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.755800 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-registration-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755831 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-hostroot\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-modprobe-d\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-systemd\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4743de47-58d7-412d-b1de-2d99d6eedad8-ovn-node-metrics-cert\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.755993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af17d5fd-f771-4b83-9885-73f28261f87c-tmp-dir\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-var-lib-kubelet\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756037 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e825faf-4777-4030-ac38-ff6f4c19cefa-tmp\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.756070 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-ovnkube-config\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da0da73e-1565-4cd1-a906-37b8316b63e6-agent-certs\") pod \"konnectivity-agent-qcf6t\" (UID: \"da0da73e-1565-4cd1-a906-37b8316b63e6\") " pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756142 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4v7x\" (UniqueName: \"kubernetes.io/projected/13995368-b3df-40c6-a914-591ec9da1ded-kube-api-access-n4v7x\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-systemd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dc2d1ac6-4fd6-4189-b443-308666150cfc-dbus\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-cni-bin\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-device-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756339 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da0da73e-1565-4cd1-a906-37b8316b63e6-konnectivity-ca\") pod \"konnectivity-agent-qcf6t\" (UID: \"da0da73e-1565-4cd1-a906-37b8316b63e6\") " pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756381 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-run-netns\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-node-log\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-system-cni-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-host\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756490 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-tuned\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.756531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756540 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-cni-netd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756566 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2xp\" (UniqueName: \"kubernetes.io/projected/671a9f0d-a582-476a-968d-44a4e93192f7-kube-api-access-pf2xp\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-cnibin\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-cni-binary-copy\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756647 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-cni-bin\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756663 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-socket-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-kubernetes\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysctl-d\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756738 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-sys\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756730 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756768 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756795 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-os-release\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51b3fa95-4da3-454c-8026-2d6e9469fae5-host-slash\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-socket-dir-parent\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-system-cni-dir\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756929 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-cnibin\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-cni-binary-copy\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.757412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.756984 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-log-socket\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757035 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-netns\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757062 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-cni-multus\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757087 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-kubelet\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-ovnkube-script-lib\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51b3fa95-4da3-454c-8026-2d6e9469fae5-iptables-alerter-script\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757169 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757250 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-conf-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757275 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-cnibin\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-daemon-config\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757299 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-socket-dir-parent\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-multus-certs\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757344 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-os-release\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757344 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-systemd-units\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757398 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-cni-bin\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757485 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51b3fa95-4da3-454c-8026-2d6e9469fae5-host-slash\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.758098 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757486 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-system-cni-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757514 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-os-release\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757537 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-system-cni-dir\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757531 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-cni-multus\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757566 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-multus-certs\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757709 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-conf-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757737 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-netns\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757761 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-cnibin\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/af17d5fd-f771-4b83-9885-73f28261f87c-hosts-file\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnd4p\" (UniqueName: \"kubernetes.io/projected/af17d5fd-f771-4b83-9885-73f28261f87c-kube-api-access-jnd4p\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysctl-conf\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxrxk\" (UniqueName: \"kubernetes.io/projected/350c83c0-715e-4925-8f3e-27ff51dee9b1-kube-api-access-qxrxk\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757902 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-cni-binary-copy\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.757968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dc2d1ac6-4fd6-4189-b443-308666150cfc-kubelet-config\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-etc-kubernetes\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758057 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13995368-b3df-40c6-a914-591ec9da1ded-cni-binary-copy\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-env-overrides\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.758721 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758111 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-etc-kubernetes\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758130 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51b3fa95-4da3-454c-8026-2d6e9469fae5-iptables-alerter-script\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758172 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-cni-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758205 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758224 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-cni-dir\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758244 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-etc-selinux\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758306 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-sys-fs\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758336 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-multus-daemon-config\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758356 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-lib-modules\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpd4v\" (UniqueName: \"kubernetes.io/projected/7e825faf-4777-4030-ac38-ff6f4c19cefa-kube-api-access-wpd4v\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758427 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-etc-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758449 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/671a9f0d-a582-476a-968d-44a4e93192f7-serviceca\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758470 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-k8s-cni-cncf-io\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758509 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-run-k8s-cni-cncf-io\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758511 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2drkw\" (UniqueName: \"kubernetes.io/projected/1c9963fa-c6e7-41d0-bfd4-f63029093704-kube-api-access-2drkw\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758540 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758563 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.759553 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758590 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/671a9f0d-a582-476a-968d-44a4e93192f7-host\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv5j\" (UniqueName: \"kubernetes.io/projected/51b3fa95-4da3-454c-8026-2d6e9469fae5-kube-api-access-kqv5j\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-kubelet\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758677 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-host-var-lib-kubelet\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysconfig\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758730 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-run\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-slash\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-var-lib-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd9pd\" (UniqueName: \"kubernetes.io/projected/4743de47-58d7-412d-b1de-2d99d6eedad8-kube-api-access-qd9pd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758869 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.758906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-ovn\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.760386 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.759095 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13995368-b3df-40c6-a914-591ec9da1ded-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.761091 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.761066 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da0da73e-1565-4cd1-a906-37b8316b63e6-agent-certs\") pod \"konnectivity-agent-qcf6t\" (UID: \"da0da73e-1565-4cd1-a906-37b8316b63e6\") " pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.765645 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.765620 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4v7x\" (UniqueName: \"kubernetes.io/projected/13995368-b3df-40c6-a914-591ec9da1ded-kube-api-access-n4v7x\") pod \"multus-additional-cni-plugins-69mr4\" (UID: \"13995368-b3df-40c6-a914-591ec9da1ded\") " pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.765741 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.765628 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6llz4\" (UniqueName: \"kubernetes.io/projected/e95e3a6e-fc04-4ed0-a54f-10ec01027c19-kube-api-access-6llz4\") pod \"multus-47vfx\" (UID: \"e95e3a6e-fc04-4ed0-a54f-10ec01027c19\") " pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.765837 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.765789 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqv5j\" (UniqueName: \"kubernetes.io/projected/51b3fa95-4da3-454c-8026-2d6e9469fae5-kube-api-access-kqv5j\") pod \"iptables-alerter-5t9nd\" (UID: \"51b3fa95-4da3-454c-8026-2d6e9469fae5\") " pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.825113 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.825050 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" event={"ID":"cafb6815aa15ff96ffbd1f08c40a3af8","Type":"ContainerStarted","Data":"9776d396da64a357a2e914f8ee986f27ed2b22bfb01596dff07ef8a4204142e5"} Jul 08 06:54:49.826496 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.826465 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" event={"ID":"44133c4c266a4c88a4c5bdd8cf3600bf","Type":"ContainerStarted","Data":"461a63ad1106e3c1b2c787f364ad399b23d0402e947611754bdc08d1b2863ac2"} Jul 08 06:54:49.859973 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.859936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-host\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.859973 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.859978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-tuned\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-cni-netd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860054 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2xp\" (UniqueName: \"kubernetes.io/projected/671a9f0d-a582-476a-968d-44a4e93192f7-kube-api-access-pf2xp\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-socket-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860090 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-host\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-kubernetes\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysctl-d\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-sys\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860155 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-cni-netd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860095 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-log-socket\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-kubelet\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860278 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-kubernetes\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-ovnkube-script-lib\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-systemd-units\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-cni-bin\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.860760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/af17d5fd-f771-4b83-9885-73f28261f87c-hosts-file\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnd4p\" (UniqueName: \"kubernetes.io/projected/af17d5fd-f771-4b83-9885-73f28261f87c-kube-api-access-jnd4p\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860406 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-socket-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysctl-conf\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860445 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-sys\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860446 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysctl-d\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860462 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxrxk\" (UniqueName: \"kubernetes.io/projected/350c83c0-715e-4925-8f3e-27ff51dee9b1-kube-api-access-qxrxk\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860466 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-cni-bin\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860491 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-log-socket\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860512 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-systemd-units\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860514 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/af17d5fd-f771-4b83-9885-73f28261f87c-hosts-file\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dc2d1ac6-4fd6-4189-b443-308666150cfc-kubelet-config\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860554 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dc2d1ac6-4fd6-4189-b443-308666150cfc-kubelet-config\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-env-overrides\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860594 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-etc-selinux\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-sys-fs\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.862883 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860658 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860595 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-run-ovn-kubernetes\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-lib-modules\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpd4v\" (UniqueName: \"kubernetes.io/projected/7e825faf-4777-4030-ac38-ff6f4c19cefa-kube-api-access-wpd4v\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860721 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-sys-fs\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860621 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-kubelet\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-etc-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/671a9f0d-a582-476a-968d-44a4e93192f7-serviceca\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860784 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-etc-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860808 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2drkw\" (UniqueName: \"kubernetes.io/projected/1c9963fa-c6e7-41d0-bfd4-f63029093704-kube-api-access-2drkw\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860820 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-lib-modules\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860832 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysctl-conf\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860876 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860916 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/671a9f0d-a582-476a-968d-44a4e93192f7-host\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysconfig\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.863614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860966 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-etc-selinux\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.860983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-run\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861008 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-slash\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861028 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/671a9f0d-a582-476a-968d-44a4e93192f7-host\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-var-lib-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861072 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd9pd\" (UniqueName: \"kubernetes.io/projected/4743de47-58d7-412d-b1de-2d99d6eedad8-kube-api-access-qd9pd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861077 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-var-lib-openvswitch\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861086 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-run\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-ovn\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861120 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-slash\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861132 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-registration-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861140 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-sysconfig\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-modprobe-d\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861186 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-ovn\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861186 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-systemd\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.861193 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861267 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-registration-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.864494 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861301 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/671a9f0d-a582-476a-968d-44a4e93192f7-serviceca\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.861309 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:54:50.361269771 +0000 UTC m=+3.074918602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861332 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-systemd\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861435 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-modprobe-d\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4743de47-58d7-412d-b1de-2d99d6eedad8-ovn-node-metrics-cert\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861554 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af17d5fd-f771-4b83-9885-73f28261f87c-tmp-dir\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861585 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-var-lib-kubelet\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e825faf-4777-4030-ac38-ff6f4c19cefa-tmp\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.861619 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-ovnkube-config\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.861669 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:50.361650727 +0000 UTC m=+3.075299543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861946 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-env-overrides\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861967 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e825faf-4777-4030-ac38-ff6f4c19cefa-var-lib-kubelet\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861990 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-ovnkube-script-lib\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.861992 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af17d5fd-f771-4b83-9885-73f28261f87c-tmp-dir\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.865277 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862060 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-systemd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862120 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dc2d1ac6-4fd6-4189-b443-308666150cfc-dbus\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-device-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-run-netns\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862182 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-run-systemd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-node-log\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862306 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c9963fa-c6e7-41d0-bfd4-f63029093704-device-dir\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862341 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-host-run-netns\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862307 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dc2d1ac6-4fd6-4189-b443-308666150cfc-dbus\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862338 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4743de47-58d7-412d-b1de-2d99d6eedad8-ovnkube-config\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.862310 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4743de47-58d7-412d-b1de-2d99d6eedad8-node-log\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.863512 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e825faf-4777-4030-ac38-ff6f4c19cefa-etc-tuned\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.864482 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e825faf-4777-4030-ac38-ff6f4c19cefa-tmp\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.865872 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.865093 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4743de47-58d7-412d-b1de-2d99d6eedad8-ovn-node-metrics-cert\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.875958 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.875511 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:54:49.875958 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.875545 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:54:49.875958 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.875563 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:49.875958 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:49.875625 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:50.375605027 +0000 UTC m=+3.089253867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:49.877452 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.877403 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2xp\" (UniqueName: \"kubernetes.io/projected/671a9f0d-a582-476a-968d-44a4e93192f7-kube-api-access-pf2xp\") pod \"node-ca-q57p9\" (UID: \"671a9f0d-a582-476a-968d-44a4e93192f7\") " pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:49.878324 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.878227 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxrxk\" (UniqueName: \"kubernetes.io/projected/350c83c0-715e-4925-8f3e-27ff51dee9b1-kube-api-access-qxrxk\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:49.879048 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.879002 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2drkw\" (UniqueName: \"kubernetes.io/projected/1c9963fa-c6e7-41d0-bfd4-f63029093704-kube-api-access-2drkw\") pod \"aws-ebs-csi-driver-node-fn6zz\" (UID: \"1c9963fa-c6e7-41d0-bfd4-f63029093704\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:49.879180 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.879148 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnd4p\" (UniqueName: \"kubernetes.io/projected/af17d5fd-f771-4b83-9885-73f28261f87c-kube-api-access-jnd4p\") pod \"node-resolver-zqjwx\" (UID: \"af17d5fd-f771-4b83-9885-73f28261f87c\") " pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:49.879663 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.879636 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd9pd\" (UniqueName: \"kubernetes.io/projected/4743de47-58d7-412d-b1de-2d99d6eedad8-kube-api-access-qd9pd\") pod \"ovnkube-node-4cknp\" (UID: \"4743de47-58d7-412d-b1de-2d99d6eedad8\") " pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:49.880008 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.879989 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpd4v\" (UniqueName: \"kubernetes.io/projected/7e825faf-4777-4030-ac38-ff6f4c19cefa-kube-api-access-wpd4v\") pod \"tuned-wp9fw\" (UID: \"7e825faf-4777-4030-ac38-ff6f4c19cefa\") " pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:49.951735 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.951526 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5t9nd" Jul 08 06:54:49.962897 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.962871 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:54:49.973000 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.972974 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47vfx" Jul 08 06:54:49.978862 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.978845 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-69mr4" Jul 08 06:54:49.994025 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:49.993991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q57p9" Jul 08 06:54:50.002047 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.002027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" Jul 08 06:54:50.011984 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.011964 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" Jul 08 06:54:50.019822 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.019802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:54:50.026904 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.026882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqjwx" Jul 08 06:54:50.062676 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.062651 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 06:54:50.365507 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.365466 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:50.365673 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.365532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:50.365673 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.365644 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:50.365673 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.365668 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:50.365825 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.365720 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:54:51.365702561 +0000 UTC m=+4.079351377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:50.365825 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.365739 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:51.365731953 +0000 UTC m=+4.079380769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:50.466683 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.466651 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:50.466847 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.466780 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:54:50.466847 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.466794 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:54:50.466847 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.466803 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:50.466972 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.466854 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:51.466841529 +0000 UTC m=+4.180490345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:50.687663 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.687569 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 06:49:50 +0000 UTC" deadline="2028-04-20 03:55:42.363959324 +0000 UTC" Jul 08 06:54:50.687663 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.687612 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15645h0m51.676351762s" Jul 08 06:54:50.819746 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:50.819714 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:50.819885 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:50.819828 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:54:50.881313 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.881256 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9963fa_c6e7_41d0_bfd4_f63029093704.slice/crio-2542f13e695e37393a586c890c74683e7cfcd633b08d0501c0df8444d20ec23e WatchSource:0}: Error finding container 2542f13e695e37393a586c890c74683e7cfcd633b08d0501c0df8444d20ec23e: Status 404 returned error can't find the container with id 2542f13e695e37393a586c890c74683e7cfcd633b08d0501c0df8444d20ec23e Jul 08 06:54:50.885011 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.884981 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e825faf_4777_4030_ac38_ff6f4c19cefa.slice/crio-d08704adc838ad5ec2a4f44c9ce7e6c296a2411c21cdb2a19d560366c5ff9a1c WatchSource:0}: Error finding container d08704adc838ad5ec2a4f44c9ce7e6c296a2411c21cdb2a19d560366c5ff9a1c: Status 404 returned error can't find the container with id d08704adc838ad5ec2a4f44c9ce7e6c296a2411c21cdb2a19d560366c5ff9a1c Jul 08 06:54:50.888079 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.888053 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0da73e_1565_4cd1_a906_37b8316b63e6.slice/crio-fc7205ad11450f3e1c8ab394530af93fea87e65e5449a374b8b86ba032e8ab19 WatchSource:0}: Error finding container fc7205ad11450f3e1c8ab394530af93fea87e65e5449a374b8b86ba032e8ab19: Status 404 returned error can't find the container with id fc7205ad11450f3e1c8ab394530af93fea87e65e5449a374b8b86ba032e8ab19 Jul 08 06:54:50.888960 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.888929 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95e3a6e_fc04_4ed0_a54f_10ec01027c19.slice/crio-1d1c04b52c348c543b908e33ef61356f44bc239fccbd0feb3ff6cf2e92e71d77 WatchSource:0}: Error finding container 1d1c04b52c348c543b908e33ef61356f44bc239fccbd0feb3ff6cf2e92e71d77: Status 404 returned error can't find the container with id 1d1c04b52c348c543b908e33ef61356f44bc239fccbd0feb3ff6cf2e92e71d77 Jul 08 06:54:50.890961 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.890902 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b3fa95_4da3_454c_8026_2d6e9469fae5.slice/crio-ec489bad9aef2427c212bddcede1648da2b8de3e956b674102a8c4a069aec4e3 WatchSource:0}: Error finding container ec489bad9aef2427c212bddcede1648da2b8de3e956b674102a8c4a069aec4e3: Status 404 returned error can't find the container with id ec489bad9aef2427c212bddcede1648da2b8de3e956b674102a8c4a069aec4e3 Jul 08 06:54:50.892365 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.892126 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4743de47_58d7_412d_b1de_2d99d6eedad8.slice/crio-aeefba175fafe4ba7e90eb08a6de29a9218e9843b5a1c3fb45475999d32d2aa5 WatchSource:0}: Error finding container aeefba175fafe4ba7e90eb08a6de29a9218e9843b5a1c3fb45475999d32d2aa5: Status 404 returned error can't find the container with id aeefba175fafe4ba7e90eb08a6de29a9218e9843b5a1c3fb45475999d32d2aa5 Jul 08 06:54:50.894316 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:54:50.893867 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf17d5fd_f771_4b83_9885_73f28261f87c.slice/crio-45ce4a993a9f29d29a88c8be8192ba4c6c2705d9ce4eda840e41ddb41fc1601b WatchSource:0}: Error finding container 45ce4a993a9f29d29a88c8be8192ba4c6c2705d9ce4eda840e41ddb41fc1601b: Status 404 returned error can't find the container with id 45ce4a993a9f29d29a88c8be8192ba4c6c2705d9ce4eda840e41ddb41fc1601b Jul 08 06:54:51.376534 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.375635 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:51.376534 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.375700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:51.376534 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.375874 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:51.376534 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.375938 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:53.375918799 +0000 UTC m=+6.089567620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:51.376534 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.376390 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:51.376534 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.376445 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:54:53.37642855 +0000 UTC m=+6.090077380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:51.485252 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.477152 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:51.485252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.477335 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:54:51.485252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.477362 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:54:51.485252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.477376 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:51.485252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.477442 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:53.477421635 +0000 UTC m=+6.191070467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:51.688604 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.688494 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 06:49:50 +0000 UTC" deadline="2027-12-19 07:03:27.870000688 +0000 UTC" Jul 08 06:54:51.688604 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.688556 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12696h8m36.181448196s" Jul 08 06:54:51.821807 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.821773 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:51.821969 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.821916 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:54:51.822419 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.822399 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:51.822592 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:51.822515 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:54:51.843597 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.843543 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" event={"ID":"cafb6815aa15ff96ffbd1f08c40a3af8","Type":"ContainerStarted","Data":"3c995c835cc39efa04eaa43a7f104ed9616e494105251f6e815b681d4a3c1dd5"} Jul 08 06:54:51.857688 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.857655 2568 generic.go:356] "Generic (PLEG): container finished" podID="44133c4c266a4c88a4c5bdd8cf3600bf" containerID="9bb447b12123799c1c135c8e11d8b07e8b12e8d4ca39b7d0ddfdc1fdb7fe71d1" exitCode=0 Jul 08 06:54:51.857847 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.857744 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" event={"ID":"44133c4c266a4c88a4c5bdd8cf3600bf","Type":"ContainerDied","Data":"9bb447b12123799c1c135c8e11d8b07e8b12e8d4ca39b7d0ddfdc1fdb7fe71d1"} Jul 08 06:54:51.859767 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.859713 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-98.ec2.internal" podStartSLOduration=2.859693852 podStartE2EDuration="2.859693852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:51.858599013 +0000 UTC m=+4.572247854" watchObservedRunningTime="2026-07-08 06:54:51.859693852 +0000 UTC m=+4.573342689" Jul 08 06:54:51.869127 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.869091 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q57p9" event={"ID":"671a9f0d-a582-476a-968d-44a4e93192f7","Type":"ContainerStarted","Data":"c19c458df80479c7bd0e04d847157f469f115fd830006c4450c56c2b0c01fb50"} Jul 08 06:54:51.877093 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.877062 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47vfx" event={"ID":"e95e3a6e-fc04-4ed0-a54f-10ec01027c19","Type":"ContainerStarted","Data":"1d1c04b52c348c543b908e33ef61356f44bc239fccbd0feb3ff6cf2e92e71d77"} Jul 08 06:54:51.880876 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.880849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" event={"ID":"7e825faf-4777-4030-ac38-ff6f4c19cefa","Type":"ContainerStarted","Data":"d08704adc838ad5ec2a4f44c9ce7e6c296a2411c21cdb2a19d560366c5ff9a1c"} Jul 08 06:54:51.892962 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.892936 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerStarted","Data":"8e98f1c0530d8349b2175594a92b1a303703cb8a96810febfcfa53ee596f74b7"} Jul 08 06:54:51.919056 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.919021 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqjwx" event={"ID":"af17d5fd-f771-4b83-9885-73f28261f87c","Type":"ContainerStarted","Data":"45ce4a993a9f29d29a88c8be8192ba4c6c2705d9ce4eda840e41ddb41fc1601b"} Jul 08 06:54:51.923080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.922977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"aeefba175fafe4ba7e90eb08a6de29a9218e9843b5a1c3fb45475999d32d2aa5"} Jul 08 06:54:51.930321 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.930255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5t9nd" event={"ID":"51b3fa95-4da3-454c-8026-2d6e9469fae5","Type":"ContainerStarted","Data":"ec489bad9aef2427c212bddcede1648da2b8de3e956b674102a8c4a069aec4e3"} Jul 08 06:54:51.936637 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.935198 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qcf6t" event={"ID":"da0da73e-1565-4cd1-a906-37b8316b63e6","Type":"ContainerStarted","Data":"fc7205ad11450f3e1c8ab394530af93fea87e65e5449a374b8b86ba032e8ab19"} Jul 08 06:54:51.944754 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:51.944683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" event={"ID":"1c9963fa-c6e7-41d0-bfd4-f63029093704","Type":"ContainerStarted","Data":"2542f13e695e37393a586c890c74683e7cfcd633b08d0501c0df8444d20ec23e"} Jul 08 06:54:52.820255 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:52.820220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:52.820757 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:52.820369 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:54:52.970468 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:52.970421 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" event={"ID":"44133c4c266a4c88a4c5bdd8cf3600bf","Type":"ContainerStarted","Data":"665cb43a1fce3227f4469b073210085d78f8a8c13e287735da8889c7179f6415"} Jul 08 06:54:53.402552 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:53.402512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:53.402740 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:53.402616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:53.402832 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.402756 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:53.402832 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.402822 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:54:57.402800752 +0000 UTC m=+10.116449596 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:53.402957 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.402897 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:53.402957 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.402934 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:57.402921546 +0000 UTC m=+10.116570366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:53.504017 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:53.503901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:53.504242 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.504130 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:54:53.504242 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.504155 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:54:53.504242 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.504185 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:53.504422 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.504253 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:57.504233559 +0000 UTC m=+10.217882394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:53.821319 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:53.821285 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:53.821810 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.821449 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:54:53.821916 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:53.821895 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:53.822047 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:53.822027 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:54:54.820327 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:54.820205 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:54.820531 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:54.820386 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:54:55.822477 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:55.821916 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:55.822477 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:55.821961 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:55.822477 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:55.822082 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:54:55.822477 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:55.822199 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:54:56.819632 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:56.819591 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:56.819835 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:56.819742 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:54:57.441405 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:57.441358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:57.441983 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:57.441429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:57.441983 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.441542 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:57.441983 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.441588 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:57.441983 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.441632 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:55:05.441609364 +0000 UTC m=+18.155258185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:54:57.441983 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.441658 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:05.441646029 +0000 UTC m=+18.155294844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:54:57.543283 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:57.542690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:57.543283 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.542967 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:54:57.543283 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.542990 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:54:57.543283 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.543003 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:57.543283 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.543065 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:05.543045212 +0000 UTC m=+18.256694032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:54:57.821007 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:57.820897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:57.821191 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.821016 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:54:57.825739 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:57.822344 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:57.825739 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:57.822707 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:54:58.820056 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:58.820025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:54:58.820554 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:58.820172 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:54:59.820241 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:59.819570 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:54:59.820241 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:59.819728 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:54:59.820241 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:54:59.819570 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:54:59.820241 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:54:59.820192 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:00.819904 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:00.819864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:00.820083 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:00.820030 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:01.819648 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:01.819610 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:01.820009 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:01.819661 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:01.820009 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:01.819763 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:01.820009 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:01.819865 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:02.819832 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:02.819797 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:02.820244 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:02.819900 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:03.819917 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:03.819884 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:03.820380 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:03.819929 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:03.820380 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:03.820014 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:03.820380 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:03.820122 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:04.820164 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:04.820123 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:04.820659 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:04.820251 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:05.502695 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:05.502652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:05.502882 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:05.502719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:05.502882 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.502811 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:55:05.502882 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.502825 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:55:05.503021 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.502886 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:55:21.502866619 +0000 UTC m=+34.216515456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:55:05.503021 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.502905 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:21.502898225 +0000 UTC m=+34.216547038 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:55:05.603710 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:05.603670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:05.603907 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.603875 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:55:05.603907 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.603901 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:55:05.604016 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.603914 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:55:05.604016 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.603980 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:21.603959627 +0000 UTC m=+34.317608442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:55:05.820059 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:05.819941 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:05.820059 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:05.819949 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:05.820573 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.820117 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:05.820573 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:05.820223 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:06.820105 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:06.820026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:06.820278 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:06.820166 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:07.821012 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:07.820761 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:07.821527 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:07.820850 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:07.821527 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:07.821108 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:07.821527 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:07.821228 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:08.819614 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:08.819578 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:08.819787 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:08.819697 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:09.819424 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:09.819388 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:09.819964 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:09.819387 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:09.819964 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:09.819529 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:09.819964 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:09.819610 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:10.820414 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:10.820148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:10.821263 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:10.820506 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:11.012401 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.012370 2568 generic.go:356] "Generic (PLEG): container finished" podID="13995368-b3df-40c6-a914-591ec9da1ded" containerID="a9085544d7a0f259ff0c8c9c3c057444947a536a3242d64061622df9e5adf8bd" exitCode=0 Jul 08 06:55:11.012612 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.012449 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerDied","Data":"a9085544d7a0f259ff0c8c9c3c057444947a536a3242d64061622df9e5adf8bd"} Jul 08 06:55:11.013925 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.013884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqjwx" event={"ID":"af17d5fd-f771-4b83-9885-73f28261f87c","Type":"ContainerStarted","Data":"6c660a8928978a17f31ba71d8991db63bc26bccea8a51539fa9f9e4b3fb2bd92"} Jul 08 06:55:11.016924 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.016903 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"1658fe39a2558fb33c83f6951bb71bae59d39cd9aac46cab1a94ef3d0a04525c"} Jul 08 06:55:11.017020 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.016975 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"7d459873ef598d8c4cad8e9e31c7e06671b125b8fdab44c428da2f1b385c021b"} Jul 08 06:55:11.017083 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.017030 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"30cfb072bb840c0d0df369f54de702db28a03ddb6308be427e6afd399fcbe386"} Jul 08 06:55:11.017083 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.017078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"a3e400ace603b0182e48e093e94c18e63a6382951dc6a005af6efdc3ab9c4efc"} Jul 08 06:55:11.017208 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.017126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"43ce426a4b8dd0fb8a7ac6cf7b6422f4801172d8018f14514816c0433b9e3a5b"} Jul 08 06:55:11.017208 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.017199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"74b0b5ed5df35092bc3d3a094692621a9df130b41babb320bb094b79c2d77470"} Jul 08 06:55:11.018344 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.018317 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qcf6t" event={"ID":"da0da73e-1565-4cd1-a906-37b8316b63e6","Type":"ContainerStarted","Data":"fb5eb4c60506055a11f70e162b5568be9d9fa383512a340c92f568b78d459fd4"} Jul 08 06:55:11.019766 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.019746 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" event={"ID":"1c9963fa-c6e7-41d0-bfd4-f63029093704","Type":"ContainerStarted","Data":"d168598150f744dd1d2aeb9e9aa31aa74f98841550a714e3aa3c08d2415158f6"} Jul 08 06:55:11.023199 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.023129 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q57p9" event={"ID":"671a9f0d-a582-476a-968d-44a4e93192f7","Type":"ContainerStarted","Data":"5b995c6bd78a407e7a69234c099421446943e76b49ba15aa2e4b88057743a72e"} Jul 08 06:55:11.024652 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.024626 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47vfx" event={"ID":"e95e3a6e-fc04-4ed0-a54f-10ec01027c19","Type":"ContainerStarted","Data":"b57f709e9107e96850ce85f5b1dc13137e0e6f0ca0cfc1bb940ea4569db55c53"} Jul 08 06:55:11.026215 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.026180 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" event={"ID":"7e825faf-4777-4030-ac38-ff6f4c19cefa","Type":"ContainerStarted","Data":"abb1aac47b0712721796172b871cc25a9f46bd91c81b9099722c26975d813409"} Jul 08 06:55:11.035270 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.035231 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-98.ec2.internal" podStartSLOduration=22.035215668 podStartE2EDuration="22.035215668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:52.987346483 +0000 UTC m=+5.700995325" watchObservedRunningTime="2026-07-08 06:55:11.035215668 +0000 UTC m=+23.748864505" Jul 08 06:55:11.050473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.050429 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wp9fw" podStartSLOduration=3.769979813 podStartE2EDuration="23.05041513s" totalImagesPullingTime="19.280435317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:11.04987012 +0000 UTC m=+23.763518958" watchObservedRunningTime="2026-07-08 06:55:11.05041513 +0000 UTC m=+23.764063965" Jul 08 06:55:11.063448 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.063411 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zqjwx" podStartSLOduration=2.8247455820000003 podStartE2EDuration="22.063402294s" totalImagesPullingTime="19.238656712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:11.063104793 +0000 UTC m=+23.776753630" watchObservedRunningTime="2026-07-08 06:55:11.063402294 +0000 UTC m=+23.777051130" Jul 08 06:55:11.075950 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.075905 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qcf6t" podStartSLOduration=7.490073218 podStartE2EDuration="23.075888941s" totalImagesPullingTime="15.585815723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:11.075743592 +0000 UTC m=+23.789392406" watchObservedRunningTime="2026-07-08 06:55:11.075888941 +0000 UTC m=+23.789537778" Jul 08 06:55:11.088970 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.088928 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q57p9" podStartSLOduration=3.855312607 podStartE2EDuration="23.088913172s" totalImagesPullingTime="19.233600565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:11.088507112 +0000 UTC m=+23.802155949" watchObservedRunningTime="2026-07-08 06:55:11.088913172 +0000 UTC m=+23.802562007" Jul 08 06:55:11.105236 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.105194 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-47vfx" podStartSLOduration=3.803488914 podStartE2EDuration="23.105181075s" totalImagesPullingTime="19.301692161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:11.104803527 +0000 UTC m=+23.818452362" watchObservedRunningTime="2026-07-08 06:55:11.105181075 +0000 UTC m=+23.818829910" Jul 08 06:55:11.659306 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.659126 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 06:55:11.731186 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.731068 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T06:55:11.659296405Z","UUID":"735c4b05-7474-4ec5-b9cd-19567cdf6efe","Handler":null,"Name":"","Endpoint":""} Jul 08 06:55:11.732838 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.732812 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 06:55:11.732974 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.732872 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 06:55:11.820311 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.820234 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:11.820499 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:11.820378 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:11.820499 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:11.820411 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:11.820908 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:11.820518 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:12.031774 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:12.031678 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" event={"ID":"1c9963fa-c6e7-41d0-bfd4-f63029093704","Type":"ContainerStarted","Data":"386fa7c0c3312181950de163bb6cf8823d5330ea3f04048f0be46f49b2b7280b"} Jul 08 06:55:12.033672 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:12.033570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5t9nd" event={"ID":"51b3fa95-4da3-454c-8026-2d6e9469fae5","Type":"ContainerStarted","Data":"e6b1359564d05ba2cb11a3888e326046daaf406386ec894d3c6c94b0c245c6db"} Jul 08 06:55:12.048677 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:12.048634 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5t9nd" podStartSLOduration=4.805863544 podStartE2EDuration="24.048619898s" totalImagesPullingTime="19.242756354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:12.048381407 +0000 UTC m=+24.762030245" watchObservedRunningTime="2026-07-08 06:55:12.048619898 +0000 UTC m=+24.762268733" Jul 08 06:55:12.569435 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:12.569402 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:55:12.819962 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:12.819902 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:12.820100 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:12.820036 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:13.043224 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:13.043185 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"62d4c60a7d62ceb45d5b8d2ac9f68461f927f4bc73bf48cb006cf8de7c538d52"} Jul 08 06:55:13.045842 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:13.045812 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" event={"ID":"1c9963fa-c6e7-41d0-bfd4-f63029093704","Type":"ContainerStarted","Data":"735f127c2b901dcc33e1f5e3b6598fcc1357c80445068346fdbdf95eecb4fd25"} Jul 08 06:55:13.820216 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:13.820181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:13.820387 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:13.820181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:13.820387 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:13.820298 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:13.820387 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:13.820359 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:14.819836 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:14.819800 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:14.820446 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:14.819930 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:15.285720 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:15.285559 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:55:15.286171 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:15.286135 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:55:15.303415 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:15.303364 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fn6zz" podStartSLOduration=5.829206138 podStartE2EDuration="27.303347113s" totalImagesPullingTime="21.474140975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:13.062506719 +0000 UTC m=+25.776155554" watchObservedRunningTime="2026-07-08 06:55:15.303347113 +0000 UTC m=+28.016996025" Jul 08 06:55:15.820063 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:15.820022 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:15.820711 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:15.820141 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:15.820711 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:15.820193 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:15.820711 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:15.820289 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:16.056359 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.056323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" event={"ID":"4743de47-58d7-412d-b1de-2d99d6eedad8","Type":"ContainerStarted","Data":"861d2575ed21d269b075c9fb2e1ad9abdf505b46ba7e0bd20ae4abaac295d5fc"} Jul 08 06:55:16.056577 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.056558 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:55:16.056654 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.056623 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:55:16.059466 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.059442 2568 generic.go:356] "Generic (PLEG): container finished" podID="13995368-b3df-40c6-a914-591ec9da1ded" containerID="fc0a6d3acf2153093262ec6b09896659f56cc981d387728150fe57939b8189dc" exitCode=0 Jul 08 06:55:16.060331 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.060298 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerDied","Data":"fc0a6d3acf2153093262ec6b09896659f56cc981d387728150fe57939b8189dc"} Jul 08 06:55:16.061223 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.061200 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qcf6t" Jul 08 06:55:16.073313 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.073266 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:55:16.083125 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.083087 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" podStartSLOduration=8.769492742 podStartE2EDuration="28.083076594s" totalImagesPullingTime="19.313583852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:16.081775805 +0000 UTC m=+28.795424642" watchObservedRunningTime="2026-07-08 06:55:16.083076594 +0000 UTC m=+28.796725429" Jul 08 06:55:16.820196 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:16.820146 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:16.820724 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:16.820299 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:17.048205 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.047985 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r7zbh"] Jul 08 06:55:17.048341 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.048320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:17.048493 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:17.048455 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:17.048627 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.048609 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5mkr5"] Jul 08 06:55:17.051536 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.051515 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kmdj4"] Jul 08 06:55:17.051646 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.051623 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:17.051717 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:17.051701 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:17.061638 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.061609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:17.061880 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:17.061854 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:17.062266 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.062243 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:55:17.077745 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:17.077683 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:55:18.065613 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:18.065577 2568 generic.go:356] "Generic (PLEG): container finished" podID="13995368-b3df-40c6-a914-591ec9da1ded" containerID="8603d3dea35f5b19264efaefb23efbf632d8df3102b6b33d5f6aff9f1879a0c1" exitCode=0 Jul 08 06:55:18.066290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:18.065664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerDied","Data":"8603d3dea35f5b19264efaefb23efbf632d8df3102b6b33d5f6aff9f1879a0c1"} Jul 08 06:55:18.819309 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:18.819281 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:18.819475 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:18.819416 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:18.819475 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:18.819419 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:18.819475 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:18.819444 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:18.819573 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:18.819513 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:18.819614 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:18.819589 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:20.072510 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:20.072422 2568 generic.go:356] "Generic (PLEG): container finished" podID="13995368-b3df-40c6-a914-591ec9da1ded" containerID="34e1e26cab76097ba41e228b2799adb558438b55852a91872c1e547f2c5b06f5" exitCode=0 Jul 08 06:55:20.072510 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:20.072463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerDied","Data":"34e1e26cab76097ba41e228b2799adb558438b55852a91872c1e547f2c5b06f5"} Jul 08 06:55:20.819783 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:20.819591 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:20.819966 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:20.819592 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:20.819966 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:20.819874 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:20.820081 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:20.819965 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:20.820081 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:20.819591 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:20.820081 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:20.820066 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:21.527845 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:21.527810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:21.528361 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:21.527858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:21.528361 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.527969 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:55:21.528361 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.527972 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 06:55:21.528361 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.528044 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:53.528025126 +0000 UTC m=+66.241673957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 06:55:21.528361 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.528100 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret podName:dc2d1ac6-4fd6-4189-b443-308666150cfc nodeName:}" failed. No retries permitted until 2026-07-08 06:55:53.528087825 +0000 UTC m=+66.241736643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret") pod "global-pull-secret-syncer-5mkr5" (UID: "dc2d1ac6-4fd6-4189-b443-308666150cfc") : object "kube-system"/"original-pull-secret" not registered Jul 08 06:55:21.629163 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:21.629133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:21.629324 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.629281 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 06:55:21.629324 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.629296 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 06:55:21.629324 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.629306 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2crs5 for pod openshift-network-diagnostics/network-check-target-r7zbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:55:21.629501 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:21.629357 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5 podName:80dbef4c-3d50-4bf2-be0b-d110603e4fc1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:53.629342994 +0000 UTC m=+66.342991807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2crs5" (UniqueName: "kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5") pod "network-check-target-r7zbh" (UID: "80dbef4c-3d50-4bf2-be0b-d110603e4fc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 06:55:22.819404 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:22.819366 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:22.819404 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:22.819404 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:22.819920 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:22.819491 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:22.819920 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:22.819505 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:55:22.819920 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:22.819612 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5mkr5" podUID="dc2d1ac6-4fd6-4189-b443-308666150cfc" Jul 08 06:55:22.819920 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:22.819702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r7zbh" podUID="80dbef4c-3d50-4bf2-be0b-d110603e4fc1" Jul 08 06:55:22.997366 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:22.997333 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-98.ec2.internal" event="NodeReady" Jul 08 06:55:22.997541 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:22.997481 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 06:55:23.041561 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.041527 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t5vjh"] Jul 08 06:55:23.044953 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.044926 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2v9h2"] Jul 08 06:55:23.045121 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.045058 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.047283 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.047261 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 06:55:23.047392 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.047307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vxwlb\"" Jul 08 06:55:23.047392 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.047344 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.047701 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.047680 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 06:55:23.049524 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.049500 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 06:55:23.049637 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.049528 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2tvxw\"" Jul 08 06:55:23.049715 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.049697 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 06:55:23.049916 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.049902 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 06:55:23.056281 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.056253 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t5vjh"] Jul 08 06:55:23.058021 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.057985 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2v9h2"] Jul 08 06:55:23.142749 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.142719 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26c3381-ae7d-4f03-a66f-163d012fbb99-tmp-dir\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.142749 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.142757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6t66\" (UniqueName: \"kubernetes.io/projected/e26c3381-ae7d-4f03-a66f-163d012fbb99-kube-api-access-k6t66\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.142976 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.142800 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.142976 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.142879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e26c3381-ae7d-4f03-a66f-163d012fbb99-config-volume\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.244060 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.244279 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fde777f5-9d61-4513-aa36-68fc4d4cd507-cert\") pod \"ingress-canary-2v9h2\" (UID: \"fde777f5-9d61-4513-aa36-68fc4d4cd507\") " pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.244279 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e26c3381-ae7d-4f03-a66f-163d012fbb99-config-volume\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.244279 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:23.244212 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:23.244279 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26c3381-ae7d-4f03-a66f-163d012fbb99-tmp-dir\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.244279 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:23.244276 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:23.744259603 +0000 UTC m=+36.457908416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:23.244534 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6t66\" (UniqueName: \"kubernetes.io/projected/e26c3381-ae7d-4f03-a66f-163d012fbb99-kube-api-access-k6t66\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.244534 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/fde777f5-9d61-4513-aa36-68fc4d4cd507-kube-api-access-vmdjl\") pod \"ingress-canary-2v9h2\" (UID: \"fde777f5-9d61-4513-aa36-68fc4d4cd507\") " pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.244625 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244578 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26c3381-ae7d-4f03-a66f-163d012fbb99-tmp-dir\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.244822 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.244800 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e26c3381-ae7d-4f03-a66f-163d012fbb99-config-volume\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.255419 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.255362 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6t66\" (UniqueName: \"kubernetes.io/projected/e26c3381-ae7d-4f03-a66f-163d012fbb99-kube-api-access-k6t66\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.345384 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.345344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/fde777f5-9d61-4513-aa36-68fc4d4cd507-kube-api-access-vmdjl\") pod \"ingress-canary-2v9h2\" (UID: \"fde777f5-9d61-4513-aa36-68fc4d4cd507\") " pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.345555 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.345403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fde777f5-9d61-4513-aa36-68fc4d4cd507-cert\") pod \"ingress-canary-2v9h2\" (UID: \"fde777f5-9d61-4513-aa36-68fc4d4cd507\") " pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.348112 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.348090 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fde777f5-9d61-4513-aa36-68fc4d4cd507-cert\") pod \"ingress-canary-2v9h2\" (UID: \"fde777f5-9d61-4513-aa36-68fc4d4cd507\") " pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.354137 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.354106 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmdjl\" (UniqueName: \"kubernetes.io/projected/fde777f5-9d61-4513-aa36-68fc4d4cd507-kube-api-access-vmdjl\") pod \"ingress-canary-2v9h2\" (UID: \"fde777f5-9d61-4513-aa36-68fc4d4cd507\") " pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.365882 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.365860 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2v9h2" Jul 08 06:55:23.749480 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:23.749447 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:23.749666 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:23.749608 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:23.749726 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:23.749680 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:24.749659465 +0000 UTC m=+37.463308303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:24.035651 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.035623 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2v9h2"] Jul 08 06:55:24.039664 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:55:24.039632 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde777f5_9d61_4513_aa36_68fc4d4cd507.slice/crio-a15d204ef310ee22aaded805c7c7b654865e193483516e0c103b913c04ba271c WatchSource:0}: Error finding container a15d204ef310ee22aaded805c7c7b654865e193483516e0c103b913c04ba271c: Status 404 returned error can't find the container with id a15d204ef310ee22aaded805c7c7b654865e193483516e0c103b913c04ba271c Jul 08 06:55:24.085049 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.085017 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerStarted","Data":"1d1f218daa3e860136fe82474e1c0c443bf4568219ff1830ecd08e61f3abf418"} Jul 08 06:55:24.086227 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.086205 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2v9h2" event={"ID":"fde777f5-9d61-4513-aa36-68fc4d4cd507","Type":"ContainerStarted","Data":"a15d204ef310ee22aaded805c7c7b654865e193483516e0c103b913c04ba271c"} Jul 08 06:55:24.754758 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.754716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:24.754928 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:24.754861 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:24.754928 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:24.754926 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:26.754908375 +0000 UTC m=+39.468557202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:24.819769 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.819735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:24.819932 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.819735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:24.820005 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.819950 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:24.823567 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.823412 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 06:55:24.823567 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.823431 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 06:55:24.823567 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.823436 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-smjcg\"" Jul 08 06:55:24.823567 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.823433 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 06:55:24.823567 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.823465 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 06:55:24.823567 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:24.823489 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6plfv\"" Jul 08 06:55:25.091814 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:25.091725 2568 generic.go:356] "Generic (PLEG): container finished" podID="13995368-b3df-40c6-a914-591ec9da1ded" containerID="1d1f218daa3e860136fe82474e1c0c443bf4568219ff1830ecd08e61f3abf418" exitCode=0 Jul 08 06:55:25.091814 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:25.091806 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerDied","Data":"1d1f218daa3e860136fe82474e1c0c443bf4568219ff1830ecd08e61f3abf418"} Jul 08 06:55:26.097001 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:26.096964 2568 generic.go:356] "Generic (PLEG): container finished" podID="13995368-b3df-40c6-a914-591ec9da1ded" containerID="39d32950dd63a2856e89d28af9fd9ed48ac9188fddcc5c4e18245fcb6bde5ea4" exitCode=0 Jul 08 06:55:26.097481 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:26.097035 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerDied","Data":"39d32950dd63a2856e89d28af9fd9ed48ac9188fddcc5c4e18245fcb6bde5ea4"} Jul 08 06:55:26.769656 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:26.769623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:26.769811 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:26.769775 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:26.769855 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:26.769845 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:30.769828482 +0000 UTC m=+43.483477314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:27.101180 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:27.101132 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2v9h2" event={"ID":"fde777f5-9d61-4513-aa36-68fc4d4cd507","Type":"ContainerStarted","Data":"341fdc7800fd81b87f3780a47ded6e4c960592c8179b925b33e1c9e4a9221ff4"} Jul 08 06:55:27.104294 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:27.104270 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69mr4" event={"ID":"13995368-b3df-40c6-a914-591ec9da1ded","Type":"ContainerStarted","Data":"404e1376d431e0f4e54afe77a4ecfbba81387ff2cc9cd241678e5a5fe3afa614"} Jul 08 06:55:27.119680 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:27.119635 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2v9h2" podStartSLOduration=1.991996737 podStartE2EDuration="4.119620904s" totalImagesPullingTime="2.127624167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:27.118133571 +0000 UTC m=+39.831782412" watchObservedRunningTime="2026-07-08 06:55:27.119620904 +0000 UTC m=+39.833269736" Jul 08 06:55:27.141988 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:27.141852 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-69mr4" podStartSLOduration=9.851040770000001 podStartE2EDuration="39.141832551s" totalImagesPullingTime="29.290791781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:27.140668543 +0000 UTC m=+39.854317382" watchObservedRunningTime="2026-07-08 06:55:27.141832551 +0000 UTC m=+39.855481388" Jul 08 06:55:30.799576 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:30.799530 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:30.799998 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:30.799691 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:30.799998 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:30.799796 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:38.799779747 +0000 UTC m=+51.513428561 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:38.854898 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:38.854859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:38.855389 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:38.855022 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:38.855389 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:38.855109 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:54.85509144 +0000 UTC m=+67.568740253 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:49.084546 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:49.084517 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4cknp" Jul 08 06:55:53.558533 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.558495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:55:53.558952 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.558591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:53.561167 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.561139 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 06:55:53.561283 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.561266 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 06:55:53.569253 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:53.569231 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 06:55:53.569352 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:53.569306 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:57.569283432 +0000 UTC m=+130.282932252 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : secret "metrics-daemon-secret" not found Jul 08 06:55:53.572512 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.572492 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dc2d1ac6-4fd6-4189-b443-308666150cfc-original-pull-secret\") pod \"global-pull-secret-syncer-5mkr5\" (UID: \"dc2d1ac6-4fd6-4189-b443-308666150cfc\") " pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:53.645250 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.645206 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5mkr5" Jul 08 06:55:53.659519 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.659487 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:53.662344 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.662324 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 06:55:53.672767 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.672746 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 06:55:53.683532 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.683505 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2crs5\" (UniqueName: \"kubernetes.io/projected/80dbef4c-3d50-4bf2-be0b-d110603e4fc1-kube-api-access-2crs5\") pod \"network-check-target-r7zbh\" (UID: \"80dbef4c-3d50-4bf2-be0b-d110603e4fc1\") " pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:53.766855 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.766823 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5mkr5"] Jul 08 06:55:53.770028 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:55:53.770002 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2d1ac6_4fd6_4189_b443_308666150cfc.slice/crio-eedcc47646dd1f679ba1f0d9b7762e920ead42061a496c3849fe0ac333b37f94 WatchSource:0}: Error finding container eedcc47646dd1f679ba1f0d9b7762e920ead42061a496c3849fe0ac333b37f94: Status 404 returned error can't find the container with id eedcc47646dd1f679ba1f0d9b7762e920ead42061a496c3849fe0ac333b37f94 Jul 08 06:55:53.941451 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.941429 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-smjcg\"" Jul 08 06:55:53.949609 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:53.949593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:54.065719 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:54.065691 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r7zbh"] Jul 08 06:55:54.068242 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:55:54.068215 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80dbef4c_3d50_4bf2_be0b_d110603e4fc1.slice/crio-98bba76d4b5f39434c43f28079a0396d4595b0525866d895c99499ea0ab50bbc WatchSource:0}: Error finding container 98bba76d4b5f39434c43f28079a0396d4595b0525866d895c99499ea0ab50bbc: Status 404 returned error can't find the container with id 98bba76d4b5f39434c43f28079a0396d4595b0525866d895c99499ea0ab50bbc Jul 08 06:55:54.167568 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:54.167531 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r7zbh" event={"ID":"80dbef4c-3d50-4bf2-be0b-d110603e4fc1","Type":"ContainerStarted","Data":"98bba76d4b5f39434c43f28079a0396d4595b0525866d895c99499ea0ab50bbc"} Jul 08 06:55:54.168696 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:54.168670 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5mkr5" event={"ID":"dc2d1ac6-4fd6-4189-b443-308666150cfc","Type":"ContainerStarted","Data":"eedcc47646dd1f679ba1f0d9b7762e920ead42061a496c3849fe0ac333b37f94"} Jul 08 06:55:54.869809 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:54.869765 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:55:54.870300 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:54.869946 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:55:54.870300 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:55:54.870028 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:26.870006779 +0000 UTC m=+99.583655593 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:55:59.184601 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:59.184553 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r7zbh" event={"ID":"80dbef4c-3d50-4bf2-be0b-d110603e4fc1","Type":"ContainerStarted","Data":"78e657ff16e93733d6d538452d99482e6b50d7463582627e3f272701700fa9cb"} Jul 08 06:55:59.185045 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:59.184699 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:55:59.185976 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:59.185947 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5mkr5" event={"ID":"dc2d1ac6-4fd6-4189-b443-308666150cfc","Type":"ContainerStarted","Data":"725f788465af11b649b68fb1f3452e2d0965563773fb749bbc9ad2920e1359b4"} Jul 08 06:55:59.200118 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:59.200078 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r7zbh" podStartSLOduration=66.890661187 podStartE2EDuration="1m11.20006688s" totalImagesPullingTime="4.309405693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:59.19979119 +0000 UTC m=+71.913440026" watchObservedRunningTime="2026-07-08 06:55:59.20006688 +0000 UTC m=+71.913715716" Jul 08 06:55:59.213987 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:55:59.213929 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5mkr5" podStartSLOduration=66.604070151 podStartE2EDuration="1m11.213917415s" totalImagesPullingTime="4.609847264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:59.213731164 +0000 UTC m=+71.927379997" watchObservedRunningTime="2026-07-08 06:55:59.213917415 +0000 UTC m=+71.927566252" Jul 08 06:56:26.884662 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:56:26.884565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:56:26.885063 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:56:26.884732 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:56:26.885063 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:56:26.884795 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:57:30.884780817 +0000 UTC m=+163.598429631 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:56:30.191506 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:56:30.191471 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r7zbh" Jul 08 06:56:57.582128 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:56:57.582085 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:56:57.582610 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:56:57.582215 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 06:56:57.582610 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:56:57.582280 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:59.582264693 +0000 UTC m=+252.295913507 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : secret "metrics-daemon-secret" not found Jul 08 06:57:26.058819 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:57:26.058779 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t5vjh" podUID="e26c3381-ae7d-4f03-a66f-163d012fbb99" Jul 08 06:57:26.386765 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:57:26.386687 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t5vjh" Jul 08 06:57:27.832673 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:57:27.832638 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:57:30.900660 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:57:30.900607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:57:30.901119 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:57:30.900755 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 06:57:30.901119 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:57:30.900825 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls podName:e26c3381-ae7d-4f03-a66f-163d012fbb99 nodeName:}" failed. No retries permitted until 2026-07-08 06:59:32.900808032 +0000 UTC m=+285.614456850 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls") pod "dns-default-t5vjh" (UID: "e26c3381-ae7d-4f03-a66f-163d012fbb99") : secret "dns-default-metrics-tls" not found Jul 08 06:57:41.820116 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:57:41.820079 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:58:45.149968 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.149938 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27"] Jul 08 06:58:45.151661 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.151645 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" Jul 08 06:58:45.153976 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.153954 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zlw2f\"" Jul 08 06:58:45.154177 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.153998 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 06:58:45.154814 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.154797 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 06:58:45.170477 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.170454 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-7bpgm"] Jul 08 06:58:45.172005 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.171988 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth"] Jul 08 06:58:45.172117 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.172102 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.173665 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.173644 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.176523 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.176508 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 06:58:45.176523 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.176521 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8mbsx\"" Jul 08 06:58:45.176991 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.176972 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 06:58:45.177094 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.176995 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 06:58:45.177094 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.176978 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 06:58:45.178050 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.178031 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 06:58:45.178135 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.178060 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 06:58:45.178135 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.178075 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 06:58:45.178135 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.178121 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 06:58:45.179427 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.179294 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-np4wp\"" Jul 08 06:58:45.182959 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.182943 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 06:58:45.195012 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.194990 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth"] Jul 08 06:58:45.195893 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.195873 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-7bpgm"] Jul 08 06:58:45.211230 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.211201 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27"] Jul 08 06:58:45.229836 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.229811 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-85bc4dbc77-fklwm"] Jul 08 06:58:45.231419 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.231405 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.233996 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.233973 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 06:58:45.234090 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.234001 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 06:58:45.234346 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.234330 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 06:58:45.234405 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.234365 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 06:58:45.234487 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.234473 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 06:58:45.234771 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.234756 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 06:58:45.235214 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.235202 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d6vv4\"" Jul 08 06:58:45.256628 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.256587 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85bc4dbc77-fklwm"] Jul 08 06:58:45.275287 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69brj\" (UniqueName: \"kubernetes.io/projected/018bbf07-db34-465e-a565-5cb929e86098-kube-api-access-69brj\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.275422 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eccceb79-1a81-4fda-8f22-976288fe4e6b-config\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.275422 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxwr\" (UniqueName: \"kubernetes.io/projected/219164e5-2321-4c35-b812-e9a244954db8-kube-api-access-rbxwr\") pod \"volume-data-source-validator-5d456fdbcd-8qs27\" (UID: \"219164e5-2321-4c35-b812-e9a244954db8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" Jul 08 06:58:45.275422 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275387 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-default-certificate\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.275422 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eccceb79-1a81-4fda-8f22-976288fe4e6b-trusted-ca\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/018bbf07-db34-465e-a565-5cb929e86098-config\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275460 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd7tz\" (UniqueName: \"kubernetes.io/projected/75038f86-0af0-4e4c-9a38-40c81032fa3d-kube-api-access-vd7tz\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/018bbf07-db34-465e-a565-5cb929e86098-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275533 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275549 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-stats-auth\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.275571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eccceb79-1a81-4fda-8f22-976288fe4e6b-serving-cert\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.275811 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.275584 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtg8\" (UniqueName: \"kubernetes.io/projected/eccceb79-1a81-4fda-8f22-976288fe4e6b-kube-api-access-8jtg8\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.376543 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376500 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxwr\" (UniqueName: \"kubernetes.io/projected/219164e5-2321-4c35-b812-e9a244954db8-kube-api-access-rbxwr\") pod \"volume-data-source-validator-5d456fdbcd-8qs27\" (UID: \"219164e5-2321-4c35-b812-e9a244954db8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" Jul 08 06:58:45.376543 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-default-certificate\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.376769 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eccceb79-1a81-4fda-8f22-976288fe4e6b-trusted-ca\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.376769 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/018bbf07-db34-465e-a565-5cb929e86098-config\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.376769 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd7tz\" (UniqueName: \"kubernetes.io/projected/75038f86-0af0-4e4c-9a38-40c81032fa3d-kube-api-access-vd7tz\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.376921 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376866 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/018bbf07-db34-465e-a565-5cb929e86098-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.376921 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.377024 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.377024 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-stats-auth\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.377024 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.376993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eccceb79-1a81-4fda-8f22-976288fe4e6b-serving-cert\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.377024 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.377017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtg8\" (UniqueName: \"kubernetes.io/projected/eccceb79-1a81-4fda-8f22-976288fe4e6b-kube-api-access-8jtg8\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.377252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:45.377021 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 06:58:45.377252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:45.377057 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:45.877034124 +0000 UTC m=+238.590682966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : configmap references non-existent config key: service-ca.crt Jul 08 06:58:45.377252 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:45.377220 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:45.877200003 +0000 UTC m=+238.590848818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : secret "router-metrics-certs-default" not found Jul 08 06:58:45.377419 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.377259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69brj\" (UniqueName: \"kubernetes.io/projected/018bbf07-db34-465e-a565-5cb929e86098-kube-api-access-69brj\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.377419 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.377306 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eccceb79-1a81-4fda-8f22-976288fe4e6b-config\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.377514 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.377452 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/018bbf07-db34-465e-a565-5cb929e86098-config\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.377698 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.377676 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eccceb79-1a81-4fda-8f22-976288fe4e6b-trusted-ca\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.377869 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.377852 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eccceb79-1a81-4fda-8f22-976288fe4e6b-config\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.379322 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.379301 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-default-certificate\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.379424 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.379386 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/018bbf07-db34-465e-a565-5cb929e86098-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.379729 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.379702 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eccceb79-1a81-4fda-8f22-976288fe4e6b-serving-cert\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.379812 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.379794 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-stats-auth\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.386645 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.386618 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69brj\" (UniqueName: \"kubernetes.io/projected/018bbf07-db34-465e-a565-5cb929e86098-kube-api-access-69brj\") pod \"service-ca-operator-5d46d9ffd4-gtqth\" (UID: \"018bbf07-db34-465e-a565-5cb929e86098\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.387605 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.387584 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxwr\" (UniqueName: \"kubernetes.io/projected/219164e5-2321-4c35-b812-e9a244954db8-kube-api-access-rbxwr\") pod \"volume-data-source-validator-5d456fdbcd-8qs27\" (UID: \"219164e5-2321-4c35-b812-e9a244954db8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" Jul 08 06:58:45.387691 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.387586 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtg8\" (UniqueName: \"kubernetes.io/projected/eccceb79-1a81-4fda-8f22-976288fe4e6b-kube-api-access-8jtg8\") pod \"console-operator-68b757fb67-7bpgm\" (UID: \"eccceb79-1a81-4fda-8f22-976288fe4e6b\") " pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.387884 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.387868 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd7tz\" (UniqueName: \"kubernetes.io/projected/75038f86-0af0-4e4c-9a38-40c81032fa3d-kube-api-access-vd7tz\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.460268 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.460238 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" Jul 08 06:58:45.482174 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.482132 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:45.487084 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.487065 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" Jul 08 06:58:45.605719 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.605688 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27"] Jul 08 06:58:45.608704 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:58:45.608659 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219164e5_2321_4c35_b812_e9a244954db8.slice/crio-1022cefbf878313c3fab39b1738951e6fe24c9936833aa1257d811b67664072e WatchSource:0}: Error finding container 1022cefbf878313c3fab39b1738951e6fe24c9936833aa1257d811b67664072e: Status 404 returned error can't find the container with id 1022cefbf878313c3fab39b1738951e6fe24c9936833aa1257d811b67664072e Jul 08 06:58:45.623231 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.623204 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-7bpgm"] Jul 08 06:58:45.625872 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:58:45.625848 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccceb79_1a81_4fda_8f22_976288fe4e6b.slice/crio-0179bebd98712c551a401174d0fc2fca6b3237022519677329a869a4a52a2440 WatchSource:0}: Error finding container 0179bebd98712c551a401174d0fc2fca6b3237022519677329a869a4a52a2440: Status 404 returned error can't find the container with id 0179bebd98712c551a401174d0fc2fca6b3237022519677329a869a4a52a2440 Jul 08 06:58:45.637760 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.637737 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth"] Jul 08 06:58:45.640571 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:58:45.640547 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018bbf07_db34_465e_a565_5cb929e86098.slice/crio-f8a2356b484046932c57518417655435776ed28dd22f670866e92617ce582123 WatchSource:0}: Error finding container f8a2356b484046932c57518417655435776ed28dd22f670866e92617ce582123: Status 404 returned error can't find the container with id f8a2356b484046932c57518417655435776ed28dd22f670866e92617ce582123 Jul 08 06:58:45.881326 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.881239 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.881326 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:45.881269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:45.881550 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:45.881445 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:46.881423888 +0000 UTC m=+239.595072702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : configmap references non-existent config key: service-ca.crt Jul 08 06:58:45.881550 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:45.881442 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 06:58:45.881550 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:45.881491 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:46.881480181 +0000 UTC m=+239.595128995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : secret "router-metrics-certs-default" not found Jul 08 06:58:46.578754 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:46.578715 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" event={"ID":"018bbf07-db34-465e-a565-5cb929e86098","Type":"ContainerStarted","Data":"f8a2356b484046932c57518417655435776ed28dd22f670866e92617ce582123"} Jul 08 06:58:46.580566 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:46.580536 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" event={"ID":"219164e5-2321-4c35-b812-e9a244954db8","Type":"ContainerStarted","Data":"1022cefbf878313c3fab39b1738951e6fe24c9936833aa1257d811b67664072e"} Jul 08 06:58:46.583624 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:46.583591 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" event={"ID":"eccceb79-1a81-4fda-8f22-976288fe4e6b","Type":"ContainerStarted","Data":"0179bebd98712c551a401174d0fc2fca6b3237022519677329a869a4a52a2440"} Jul 08 06:58:46.891463 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:46.891372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:46.891463 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:46.891418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:46.891692 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:46.891559 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 06:58:46.891692 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:46.891560 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:48.891541125 +0000 UTC m=+241.605189939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : configmap references non-existent config key: service-ca.crt Jul 08 06:58:46.891692 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:46.891620 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:48.891605003 +0000 UTC m=+241.605253822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : secret "router-metrics-certs-default" not found Jul 08 06:58:47.587975 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:47.587936 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" event={"ID":"219164e5-2321-4c35-b812-e9a244954db8","Type":"ContainerStarted","Data":"2f25ad045f9be1509b8c52e83dfaf7575f92420b4a5801b2fef69e3017e080e2"} Jul 08 06:58:47.603479 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:47.603425 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8qs27" podStartSLOduration=1.507607931 podStartE2EDuration="2.603407772s" totalImagesPullingTime="1.095799841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:58:47.60246774 +0000 UTC m=+240.316116579" watchObservedRunningTime="2026-07-08 06:58:47.603407772 +0000 UTC m=+240.317056614" Jul 08 06:58:48.591905 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.591803 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" event={"ID":"018bbf07-db34-465e-a565-5cb929e86098","Type":"ContainerStarted","Data":"2b7233648d686106ad99c7aa0455b81d03992271af8afa098bc6dfdd09c26284"} Jul 08 06:58:48.593406 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.593384 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/0.log" Jul 08 06:58:48.593502 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.593425 2568 generic.go:356] "Generic (PLEG): container finished" podID="eccceb79-1a81-4fda-8f22-976288fe4e6b" containerID="774d910b28f7b75472c0f2f2f288f536df3217a129a92f388764c11508ca519c" exitCode=255 Jul 08 06:58:48.593502 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.593488 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" event={"ID":"eccceb79-1a81-4fda-8f22-976288fe4e6b","Type":"ContainerDied","Data":"774d910b28f7b75472c0f2f2f288f536df3217a129a92f388764c11508ca519c"} Jul 08 06:58:48.593710 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.593697 2568 scope.go:117] "RemoveContainer" containerID="774d910b28f7b75472c0f2f2f288f536df3217a129a92f388764c11508ca519c" Jul 08 06:58:48.607828 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.607775 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" podStartSLOduration=0.943686599 podStartE2EDuration="3.607754051s" totalImagesPullingTime="2.664067452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:58:48.606605178 +0000 UTC m=+241.320254015" watchObservedRunningTime="2026-07-08 06:58:48.607754051 +0000 UTC m=+241.321402885" Jul 08 06:58:48.910420 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.910329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:48.910420 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:48.910366 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:48.910607 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:48.910511 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:52.910490638 +0000 UTC m=+245.624139464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : configmap references non-existent config key: service-ca.crt Jul 08 06:58:48.910607 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:48.910539 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 06:58:48.910607 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:48.910593 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:58:52.910577474 +0000 UTC m=+245.624226304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : secret "router-metrics-certs-default" not found Jul 08 06:58:49.215950 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.215915 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l6766"] Jul 08 06:58:49.217809 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.217789 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.220903 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.220875 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 06:58:49.221033 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.221012 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 06:58:49.221106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.221036 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 06:58:49.221290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.221275 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 06:58:49.225285 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.225268 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qvkm5\"" Jul 08 06:58:49.235857 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.235838 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l6766"] Jul 08 06:58:49.313296 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.313256 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7dzz\" (UniqueName: \"kubernetes.io/projected/d74a50ee-05f3-4bdc-b128-575325472098-kube-api-access-l7dzz\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.313480 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.313314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d74a50ee-05f3-4bdc-b128-575325472098-crio-socket\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.313480 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.313382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d74a50ee-05f3-4bdc-b128-575325472098-data-volume\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.313480 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.313409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.313698 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.313482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.413869 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.413827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.414059 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.413968 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 06:58:49.414059 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.414013 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7dzz\" (UniqueName: \"kubernetes.io/projected/d74a50ee-05f3-4bdc-b128-575325472098-kube-api-access-l7dzz\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.414059 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.414031 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:49.914015414 +0000 UTC m=+242.627664228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : secret "insights-runtime-extractor-tls" not found Jul 08 06:58:49.414248 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.414064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d74a50ee-05f3-4bdc-b128-575325472098-crio-socket\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.414248 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.414091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d74a50ee-05f3-4bdc-b128-575325472098-data-volume\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.414248 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.414112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.414248 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.414149 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d74a50ee-05f3-4bdc-b128-575325472098-crio-socket\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.414248 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.414211 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 06:58:49.414248 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.414245 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:49.91423593 +0000 UTC m=+242.627884744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : configmap "kube-rbac-proxy" not found Jul 08 06:58:49.414532 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.414438 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d74a50ee-05f3-4bdc-b128-575325472098-data-volume\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.422337 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.422319 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7dzz\" (UniqueName: \"kubernetes.io/projected/d74a50ee-05f3-4bdc-b128-575325472098-kube-api-access-l7dzz\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.599019 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.598943 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/1.log" Jul 08 06:58:49.599443 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.599402 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/0.log" Jul 08 06:58:49.599516 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.599441 2568 generic.go:356] "Generic (PLEG): container finished" podID="eccceb79-1a81-4fda-8f22-976288fe4e6b" containerID="f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7" exitCode=255 Jul 08 06:58:49.599565 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.599525 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" event={"ID":"eccceb79-1a81-4fda-8f22-976288fe4e6b","Type":"ContainerDied","Data":"f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7"} Jul 08 06:58:49.599610 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.599582 2568 scope.go:117] "RemoveContainer" containerID="774d910b28f7b75472c0f2f2f288f536df3217a129a92f388764c11508ca519c" Jul 08 06:58:49.599817 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.599793 2568 scope.go:117] "RemoveContainer" containerID="f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7" Jul 08 06:58:49.600001 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.599979 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-7bpgm_openshift-console-operator(eccceb79-1a81-4fda-8f22-976288fe4e6b)\"" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" podUID="eccceb79-1a81-4fda-8f22-976288fe4e6b" Jul 08 06:58:49.918097 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.918006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.918097 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:49.918050 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:49.918311 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.918129 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 06:58:49.918311 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.918136 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 06:58:49.918311 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.918223 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:50.918201663 +0000 UTC m=+243.631850477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : configmap "kube-rbac-proxy" not found Jul 08 06:58:49.918311 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:49.918252 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:50.918235851 +0000 UTC m=+243.631884676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : secret "insights-runtime-extractor-tls" not found Jul 08 06:58:50.603609 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:50.603584 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/1.log" Jul 08 06:58:50.604028 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:50.603884 2568 scope.go:117] "RemoveContainer" containerID="f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7" Jul 08 06:58:50.604070 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:50.604028 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-7bpgm_openshift-console-operator(eccceb79-1a81-4fda-8f22-976288fe4e6b)\"" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" podUID="eccceb79-1a81-4fda-8f22-976288fe4e6b" Jul 08 06:58:50.926489 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:50.926403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:50.926489 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:50.926454 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:50.926670 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:50.926524 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 06:58:50.926670 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:50.926545 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 06:58:50.926670 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:50.926590 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:52.926572912 +0000 UTC m=+245.640221746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : configmap "kube-rbac-proxy" not found Jul 08 06:58:50.926765 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:50.926682 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:52.926672139 +0000 UTC m=+245.640320957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : secret "insights-runtime-extractor-tls" not found Jul 08 06:58:51.384136 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:51.384108 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqjwx_af17d5fd-f771-4b83-9885-73f28261f87c/dns-node-resolver/0.log" Jul 08 06:58:52.382104 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.382075 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q57p9_671a9f0d-a582-476a-968d-44a4e93192f7/node-ca/0.log" Jul 08 06:58:52.780831 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.780805 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2v9h2_fde777f5-9d61-4513-aa36-68fc4d4cd507/serve-healthcheck-canary/0.log" Jul 08 06:58:52.882517 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.882485 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xsz4q"] Jul 08 06:58:52.884824 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.884807 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:52.887120 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.887097 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 06:58:52.887248 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.887097 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 06:58:52.888143 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.888120 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 06:58:52.888243 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.888125 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fbm2p\"" Jul 08 06:58:52.888243 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.888178 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 06:58:52.893473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.893449 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xsz4q"] Jul 08 06:58:52.943644 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:52.943817 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:52.943817 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943701 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckzwm\" (UniqueName: \"kubernetes.io/projected/3ea062be-d138-4c58-8cbe-e2797a2f42e6-kube-api-access-ckzwm\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:52.943817 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943733 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:52.943817 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943742 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 06:58:52.943817 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3ea062be-d138-4c58-8cbe-e2797a2f42e6-signing-key\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943849 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:56.943833974 +0000 UTC m=+249.657482788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : configmap "kube-rbac-proxy" not found Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943849 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943859 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:52.943877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3ea062be-d138-4c58-8cbe-e2797a2f42e6-signing-cabundle\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943908 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:59:00.943890189 +0000 UTC m=+253.657539014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : configmap references non-existent config key: service-ca.crt Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943952 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:58:56.943928193 +0000 UTC m=+249.657577020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : secret "insights-runtime-extractor-tls" not found Jul 08 06:58:52.944022 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:52.943979 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs podName:75038f86-0af0-4e4c-9a38-40c81032fa3d nodeName:}" failed. No retries permitted until 2026-07-08 06:59:00.943968799 +0000 UTC m=+253.657617616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs") pod "router-default-85bc4dbc77-fklwm" (UID: "75038f86-0af0-4e4c-9a38-40c81032fa3d") : secret "router-metrics-certs-default" not found Jul 08 06:58:53.045336 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.045260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckzwm\" (UniqueName: \"kubernetes.io/projected/3ea062be-d138-4c58-8cbe-e2797a2f42e6-kube-api-access-ckzwm\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.045336 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.045318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3ea062be-d138-4c58-8cbe-e2797a2f42e6-signing-key\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.045336 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.045338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3ea062be-d138-4c58-8cbe-e2797a2f42e6-signing-cabundle\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.045967 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.045944 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3ea062be-d138-4c58-8cbe-e2797a2f42e6-signing-cabundle\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.047759 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.047735 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3ea062be-d138-4c58-8cbe-e2797a2f42e6-signing-key\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.053525 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.053499 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckzwm\" (UniqueName: \"kubernetes.io/projected/3ea062be-d138-4c58-8cbe-e2797a2f42e6-kube-api-access-ckzwm\") pod \"service-ca-758f599c5-xsz4q\" (UID: \"3ea062be-d138-4c58-8cbe-e2797a2f42e6\") " pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.194084 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.194047 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xsz4q" Jul 08 06:58:53.315771 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.315668 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xsz4q"] Jul 08 06:58:53.318063 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:58:53.318037 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea062be_d138_4c58_8cbe_e2797a2f42e6.slice/crio-66a1fd73f0f5e5cf1f9a28b7c5d4a0716fcdf5efda82c7fab1571d0f80922a05 WatchSource:0}: Error finding container 66a1fd73f0f5e5cf1f9a28b7c5d4a0716fcdf5efda82c7fab1571d0f80922a05: Status 404 returned error can't find the container with id 66a1fd73f0f5e5cf1f9a28b7c5d4a0716fcdf5efda82c7fab1571d0f80922a05 Jul 08 06:58:53.614505 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.614416 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xsz4q" event={"ID":"3ea062be-d138-4c58-8cbe-e2797a2f42e6","Type":"ContainerStarted","Data":"95a2a9b584cb000f3a29e6452676a3174cd6eb33ca0dfe50c370ebbc094b413b"} Jul 08 06:58:53.614505 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.614460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xsz4q" event={"ID":"3ea062be-d138-4c58-8cbe-e2797a2f42e6","Type":"ContainerStarted","Data":"66a1fd73f0f5e5cf1f9a28b7c5d4a0716fcdf5efda82c7fab1571d0f80922a05"} Jul 08 06:58:53.631692 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:53.631554 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xsz4q" podStartSLOduration=1.631535477 podStartE2EDuration="1.631535477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:58:53.631128171 +0000 UTC m=+246.344777009" watchObservedRunningTime="2026-07-08 06:58:53.631535477 +0000 UTC m=+246.345184316" Jul 08 06:58:55.482692 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:55.482605 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:55.482692 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:55.482661 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:58:55.483069 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:55.482980 2568 scope.go:117] "RemoveContainer" containerID="f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7" Jul 08 06:58:55.483130 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:55.483112 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-7bpgm_openshift-console-operator(eccceb79-1a81-4fda-8f22-976288fe4e6b)\"" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" podUID="eccceb79-1a81-4fda-8f22-976288fe4e6b" Jul 08 06:58:56.979222 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:56.979185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:56.979222 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:56.979221 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:58:56.979742 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:56.979302 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 06:58:56.979742 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:56.979339 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 06:58:56.979742 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:56.979369 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:59:04.979353791 +0000 UTC m=+257.693002605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : configmap "kube-rbac-proxy" not found Jul 08 06:58:56.979742 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:56.979386 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:59:04.979376771 +0000 UTC m=+257.693025585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : secret "insights-runtime-extractor-tls" not found Jul 08 06:58:59.599594 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:58:59.599548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:58:59.600008 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:59.599735 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 06:58:59.600008 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:58:59.599813 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs podName:350c83c0-715e-4925-8f3e-27ff51dee9b1 nodeName:}" failed. No retries permitted until 2026-07-08 07:01:01.599791868 +0000 UTC m=+374.313440685 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs") pod "network-metrics-daemon-kmdj4" (UID: "350c83c0-715e-4925-8f3e-27ff51dee9b1") : secret "metrics-daemon-secret" not found Jul 08 06:59:01.010669 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.010616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:01.010669 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.010671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:01.011269 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.011247 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75038f86-0af0-4e4c-9a38-40c81032fa3d-service-ca-bundle\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:01.013001 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.012978 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75038f86-0af0-4e4c-9a38-40c81032fa3d-metrics-certs\") pod \"router-default-85bc4dbc77-fklwm\" (UID: \"75038f86-0af0-4e4c-9a38-40c81032fa3d\") " pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:01.142446 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.142415 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d6vv4\"" Jul 08 06:59:01.150536 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.150516 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:01.273978 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.273946 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85bc4dbc77-fklwm"] Jul 08 06:59:01.276286 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:01.276260 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75038f86_0af0_4e4c_9a38_40c81032fa3d.slice/crio-5a2216787dcf96425c28c3cf72b82c1a1ef8197b2f77f00c607c852ca13482d4 WatchSource:0}: Error finding container 5a2216787dcf96425c28c3cf72b82c1a1ef8197b2f77f00c607c852ca13482d4: Status 404 returned error can't find the container with id 5a2216787dcf96425c28c3cf72b82c1a1ef8197b2f77f00c607c852ca13482d4 Jul 08 06:59:01.640649 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.640560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" event={"ID":"75038f86-0af0-4e4c-9a38-40c81032fa3d","Type":"ContainerStarted","Data":"fa39fdf3183f63e2bd321ffab00b40962fe30ec54a85dc24f7dd1fd7a0e95b8b"} Jul 08 06:59:01.640649 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.640623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" event={"ID":"75038f86-0af0-4e4c-9a38-40c81032fa3d","Type":"ContainerStarted","Data":"5a2216787dcf96425c28c3cf72b82c1a1ef8197b2f77f00c607c852ca13482d4"} Jul 08 06:59:01.660377 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:01.660331 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" podStartSLOduration=16.660316485 podStartE2EDuration="16.660316485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:01.658851289 +0000 UTC m=+254.372500126" watchObservedRunningTime="2026-07-08 06:59:01.660316485 +0000 UTC m=+254.373965323" Jul 08 06:59:02.150910 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:02.150873 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:02.153581 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:02.153556 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:02.643248 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:02.643226 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:02.644568 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:02.644549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-85bc4dbc77-fklwm" Jul 08 06:59:05.045351 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:05.045319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:59:05.045351 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:05.045352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:59:05.045786 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:05.045474 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 06:59:05.045786 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:05.045540 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls podName:d74a50ee-05f3-4bdc-b128-575325472098 nodeName:}" failed. No retries permitted until 2026-07-08 06:59:21.045523757 +0000 UTC m=+273.759172571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l6766" (UID: "d74a50ee-05f3-4bdc-b128-575325472098") : secret "insights-runtime-extractor-tls" not found Jul 08 06:59:05.046002 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:05.045983 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d74a50ee-05f3-4bdc-b128-575325472098-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:59:07.821260 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:07.821228 2568 scope.go:117] "RemoveContainer" containerID="f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7" Jul 08 06:59:08.663243 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:08.663219 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 06:59:08.663591 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:08.663577 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/1.log" Jul 08 06:59:08.663638 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:08.663611 2568 generic.go:356] "Generic (PLEG): container finished" podID="eccceb79-1a81-4fda-8f22-976288fe4e6b" containerID="26eb604d17baf27adabf0064a452be1f0765f3f854401a7b071f6539c086cb7f" exitCode=255 Jul 08 06:59:08.663690 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:08.663676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" event={"ID":"eccceb79-1a81-4fda-8f22-976288fe4e6b","Type":"ContainerDied","Data":"26eb604d17baf27adabf0064a452be1f0765f3f854401a7b071f6539c086cb7f"} Jul 08 06:59:08.663745 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:08.663725 2568 scope.go:117] "RemoveContainer" containerID="f21f812181110b4e1c79bef7c95ab9ed7e503bedccbedd530320fd91b5ea35e7" Jul 08 06:59:08.664008 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:08.663988 2568 scope.go:117] "RemoveContainer" containerID="26eb604d17baf27adabf0064a452be1f0765f3f854401a7b071f6539c086cb7f" Jul 08 06:59:08.664188 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:08.664144 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-7bpgm_openshift-console-operator(eccceb79-1a81-4fda-8f22-976288fe4e6b)\"" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" podUID="eccceb79-1a81-4fda-8f22-976288fe4e6b" Jul 08 06:59:09.668303 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:09.668226 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 06:59:15.483282 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.483252 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:59:15.483674 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.483305 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:59:15.483674 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.483650 2568 scope.go:117] "RemoveContainer" containerID="26eb604d17baf27adabf0064a452be1f0765f3f854401a7b071f6539c086cb7f" Jul 08 06:59:15.483880 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:15.483852 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-7bpgm_openshift-console-operator(eccceb79-1a81-4fda-8f22-976288fe4e6b)\"" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" podUID="eccceb79-1a81-4fda-8f22-976288fe4e6b" Jul 08 06:59:15.485963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.485939 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56986c54d8-scx9h"] Jul 08 06:59:15.488782 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.488767 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.491500 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.491479 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 06:59:15.491638 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.491617 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 06:59:15.491712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.491698 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5p27c\"" Jul 08 06:59:15.492461 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.492441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 06:59:15.496702 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.496684 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 06:59:15.503844 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.503819 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56986c54d8-scx9h"] Jul 08 06:59:15.625096 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-registry-tls\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625096 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baaebca7-e610-436f-b708-ebe0ed54fbb7-trusted-ca\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625096 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625072 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7444\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-kube-api-access-m7444\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625411 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baaebca7-e610-436f-b708-ebe0ed54fbb7-ca-trust-extracted\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625411 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625293 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baaebca7-e610-436f-b708-ebe0ed54fbb7-registry-certificates\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625411 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baaebca7-e610-436f-b708-ebe0ed54fbb7-image-registry-private-configuration\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625558 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baaebca7-e610-436f-b708-ebe0ed54fbb7-installation-pull-secrets\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.625558 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.625458 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-bound-sa-token\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.726749 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.726710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7444\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-kube-api-access-m7444\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.726749 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.726748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baaebca7-e610-436f-b708-ebe0ed54fbb7-ca-trust-extracted\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.726990 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.726883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baaebca7-e610-436f-b708-ebe0ed54fbb7-registry-certificates\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.726990 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.726961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baaebca7-e610-436f-b708-ebe0ed54fbb7-image-registry-private-configuration\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.727085 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.727019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baaebca7-e610-436f-b708-ebe0ed54fbb7-installation-pull-secrets\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.727085 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.727064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-bound-sa-token\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.727206 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.727100 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baaebca7-e610-436f-b708-ebe0ed54fbb7-ca-trust-extracted\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.727206 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.727101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-registry-tls\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.727206 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.727188 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baaebca7-e610-436f-b708-ebe0ed54fbb7-trusted-ca\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.727752 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.727719 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baaebca7-e610-436f-b708-ebe0ed54fbb7-registry-certificates\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.728344 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.728324 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baaebca7-e610-436f-b708-ebe0ed54fbb7-trusted-ca\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.729352 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.729325 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baaebca7-e610-436f-b708-ebe0ed54fbb7-image-registry-private-configuration\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.729352 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.729348 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baaebca7-e610-436f-b708-ebe0ed54fbb7-installation-pull-secrets\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.729671 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.729651 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-registry-tls\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.734336 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.734311 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-bound-sa-token\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.734457 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.734432 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7444\" (UniqueName: \"kubernetes.io/projected/baaebca7-e610-436f-b708-ebe0ed54fbb7-kube-api-access-m7444\") pod \"image-registry-56986c54d8-scx9h\" (UID: \"baaebca7-e610-436f-b708-ebe0ed54fbb7\") " pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.797949 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.797923 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:15.924597 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:15.924567 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56986c54d8-scx9h"] Jul 08 06:59:15.927588 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:15.927561 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaaebca7_e610_436f_b708_ebe0ed54fbb7.slice/crio-a45dcb52fa3ff1b0e3055264db2442cbe834f858195f8b2c2f2e1781ba0c6ba2 WatchSource:0}: Error finding container a45dcb52fa3ff1b0e3055264db2442cbe834f858195f8b2c2f2e1781ba0c6ba2: Status 404 returned error can't find the container with id a45dcb52fa3ff1b0e3055264db2442cbe834f858195f8b2c2f2e1781ba0c6ba2 Jul 08 06:59:16.695203 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:16.695169 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" event={"ID":"baaebca7-e610-436f-b708-ebe0ed54fbb7","Type":"ContainerStarted","Data":"f5ce56db96516448d844e698156c514309e98299be095d4044578f1271b6b336"} Jul 08 06:59:16.695704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:16.695221 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" event={"ID":"baaebca7-e610-436f-b708-ebe0ed54fbb7","Type":"ContainerStarted","Data":"a45dcb52fa3ff1b0e3055264db2442cbe834f858195f8b2c2f2e1781ba0c6ba2"} Jul 08 06:59:16.695704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:16.695301 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:16.716732 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:16.716678 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" podStartSLOduration=1.7166622550000001 podStartE2EDuration="1.716662255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:16.715453545 +0000 UTC m=+269.429102383" watchObservedRunningTime="2026-07-08 06:59:16.716662255 +0000 UTC m=+269.430311091" Jul 08 06:59:21.068610 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:21.068571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:59:21.070846 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:21.070827 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d74a50ee-05f3-4bdc-b128-575325472098-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l6766\" (UID: \"d74a50ee-05f3-4bdc-b128-575325472098\") " pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:59:21.327026 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:21.326940 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l6766" Jul 08 06:59:21.455540 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:21.455517 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l6766"] Jul 08 06:59:21.457970 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:21.457928 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74a50ee_05f3_4bdc_b128_575325472098.slice/crio-fa70c3993862e88770ff5555c543d2d65e1de0bfd83c6e37309f873ae158d7cd WatchSource:0}: Error finding container fa70c3993862e88770ff5555c543d2d65e1de0bfd83c6e37309f873ae158d7cd: Status 404 returned error can't find the container with id fa70c3993862e88770ff5555c543d2d65e1de0bfd83c6e37309f873ae158d7cd Jul 08 06:59:21.711133 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:21.711104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6766" event={"ID":"d74a50ee-05f3-4bdc-b128-575325472098","Type":"ContainerStarted","Data":"fa70c3993862e88770ff5555c543d2d65e1de0bfd83c6e37309f873ae158d7cd"} Jul 08 06:59:25.733428 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:25.733383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6766" event={"ID":"d74a50ee-05f3-4bdc-b128-575325472098","Type":"ContainerStarted","Data":"ba12cc5d4c29d318511b34c18d08a917e4d341dfb0bff8038f180bd66441ada3"} Jul 08 06:59:26.739656 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:26.739615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6766" event={"ID":"d74a50ee-05f3-4bdc-b128-575325472098","Type":"ContainerStarted","Data":"a4f3ff840d436f92c810115d813e4dd612f4aa17513bdc4e01201d5001056d89"} Jul 08 06:59:28.749146 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:28.749108 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l6766" event={"ID":"d74a50ee-05f3-4bdc-b128-575325472098","Type":"ContainerStarted","Data":"038fb4955d10d3209a86cbcdb9a97b950751277645dd78b90c40dd0028ed5da1"} Jul 08 06:59:28.767516 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:28.767474 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-l6766" podStartSLOduration=33.153635853 podStartE2EDuration="39.767460789s" totalImagesPullingTime="6.613824936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:58:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:28.766234168 +0000 UTC m=+281.479883003" watchObservedRunningTime="2026-07-08 06:59:28.767460789 +0000 UTC m=+281.481109625" Jul 08 06:59:29.387204 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:29.387141 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t5vjh" podUID="e26c3381-ae7d-4f03-a66f-163d012fbb99" Jul 08 06:59:29.752200 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:29.752149 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t5vjh" Jul 08 06:59:29.820284 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:29.820256 2568 scope.go:117] "RemoveContainer" containerID="26eb604d17baf27adabf0064a452be1f0765f3f854401a7b071f6539c086cb7f" Jul 08 06:59:30.005520 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.005430 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-swwlb"] Jul 08 06:59:30.009101 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.009072 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.012500 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.012476 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 06:59:30.012623 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.012555 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 06:59:30.012986 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.012832 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 06:59:30.013131 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.013112 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 06:59:30.013498 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.013482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xl7cp\"" Jul 08 06:59:30.013652 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.013637 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 06:59:30.013712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.013665 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 06:59:30.038332 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-accelerators-collector-config\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038474 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038338 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbcj\" (UniqueName: \"kubernetes.io/projected/27623fbf-ed7a-4647-9200-68f9f3c4eed8-kube-api-access-wkbcj\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038474 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038372 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-textfile\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038474 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-root\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038474 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-wtmp\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038690 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038508 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-tls\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038690 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038581 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27623fbf-ed7a-4647-9200-68f9f3c4eed8-metrics-client-ca\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038690 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.038690 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.038656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-sys\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139270 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27623fbf-ed7a-4647-9200-68f9f3c4eed8-metrics-client-ca\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-sys\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-accelerators-collector-config\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139377 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbcj\" (UniqueName: \"kubernetes.io/projected/27623fbf-ed7a-4647-9200-68f9f3c4eed8-kube-api-access-wkbcj\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-textfile\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-root\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139473 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-wtmp\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139831 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-tls\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139831 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-sys\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.139831 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:30.139627 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 06:59:30.139831 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:30.139683 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-tls podName:27623fbf-ed7a-4647-9200-68f9f3c4eed8 nodeName:}" failed. No retries permitted until 2026-07-08 06:59:30.639665 +0000 UTC m=+283.353313817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-tls") pod "node-exporter-swwlb" (UID: "27623fbf-ed7a-4647-9200-68f9f3c4eed8") : secret "node-exporter-tls" not found Jul 08 06:59:30.139831 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139728 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-root\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.140065 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139858 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-wtmp\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.140065 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.139972 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-accelerators-collector-config\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.140242 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.140217 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-textfile\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.140449 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.140428 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27623fbf-ed7a-4647-9200-68f9f3c4eed8-metrics-client-ca\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.141961 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.141939 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.152213 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.152186 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbcj\" (UniqueName: \"kubernetes.io/projected/27623fbf-ed7a-4647-9200-68f9f3c4eed8-kube-api-access-wkbcj\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.643326 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.643289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-tls\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.645607 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.645581 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27623fbf-ed7a-4647-9200-68f9f3c4eed8-node-exporter-tls\") pod \"node-exporter-swwlb\" (UID: \"27623fbf-ed7a-4647-9200-68f9f3c4eed8\") " pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.654263 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.654235 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-zb79f"] Jul 08 06:59:30.658456 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.658441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:30.661062 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.661041 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-49t7g\"" Jul 08 06:59:30.661189 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.661062 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 06:59:30.661363 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.661349 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 06:59:30.671928 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.671907 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-zb79f"] Jul 08 06:59:30.743828 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.743792 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cdlj\" (UniqueName: \"kubernetes.io/projected/9766fd0b-a971-4e8d-a126-01cb547a7f34-kube-api-access-9cdlj\") pod \"downloads-685bfb5f64-zb79f\" (UID: \"9766fd0b-a971-4e8d-a126-01cb547a7f34\") " pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:30.757013 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.756989 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 06:59:30.757378 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.757034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" event={"ID":"eccceb79-1a81-4fda-8f22-976288fe4e6b","Type":"ContainerStarted","Data":"2c4be3992db154f541357f7b2e186a62e20380d276cbb575f734637b25accd61"} Jul 08 06:59:30.757378 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.757327 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:59:30.762261 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.762241 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" Jul 08 06:59:30.774666 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.774624 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-7bpgm" podStartSLOduration=43.098104133 podStartE2EDuration="45.774612401s" totalImagesPullingTime="2.676508268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:30.774086821 +0000 UTC m=+283.487735657" watchObservedRunningTime="2026-07-08 06:59:30.774612401 +0000 UTC m=+283.488261237" Jul 08 06:59:30.844416 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.844383 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cdlj\" (UniqueName: \"kubernetes.io/projected/9766fd0b-a971-4e8d-a126-01cb547a7f34-kube-api-access-9cdlj\") pod \"downloads-685bfb5f64-zb79f\" (UID: \"9766fd0b-a971-4e8d-a126-01cb547a7f34\") " pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:30.852962 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.852941 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cdlj\" (UniqueName: \"kubernetes.io/projected/9766fd0b-a971-4e8d-a126-01cb547a7f34-kube-api-access-9cdlj\") pod \"downloads-685bfb5f64-zb79f\" (UID: \"9766fd0b-a971-4e8d-a126-01cb547a7f34\") " pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:30.921412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.921330 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-swwlb" Jul 08 06:59:30.931382 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:30.931352 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27623fbf_ed7a_4647_9200_68f9f3c4eed8.slice/crio-415ed5d6765c189c1616d84948e6687c9bb2b3e084f6d4c9e671231e6804334f WatchSource:0}: Error finding container 415ed5d6765c189c1616d84948e6687c9bb2b3e084f6d4c9e671231e6804334f: Status 404 returned error can't find the container with id 415ed5d6765c189c1616d84948e6687c9bb2b3e084f6d4c9e671231e6804334f Jul 08 06:59:30.967888 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:30.967840 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:31.053357 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.053324 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 06:59:31.058524 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.058497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.061017 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.060996 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 06:59:31.061279 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.061260 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 06:59:31.061735 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.061500 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 06:59:31.061735 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.061554 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 06:59:31.061735 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.061599 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 06:59:31.061735 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.061601 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fs6cf\"" Jul 08 06:59:31.062007 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.061835 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 06:59:31.062066 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.062047 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 06:59:31.062336 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.062311 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 06:59:31.062443 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.062315 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 06:59:31.076058 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.076028 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 06:59:31.107889 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.107858 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-zb79f"] Jul 08 06:59:31.110517 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:31.110492 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9766fd0b_a971_4e8d_a126_01cb547a7f34.slice/crio-4a196b6e09ded2205e0acf61ee6ce2ca457cdd83e3bc69188bc29a4c5457f3a7 WatchSource:0}: Error finding container 4a196b6e09ded2205e0acf61ee6ce2ca457cdd83e3bc69188bc29a4c5457f3a7: Status 404 returned error can't find the container with id 4a196b6e09ded2205e0acf61ee6ce2ca457cdd83e3bc69188bc29a4c5457f3a7 Jul 08 06:59:31.157223 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157223 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157423 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157256 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmc7d\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-kube-api-access-gmc7d\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157423 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157299 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-config-out\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157423 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-web-config\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157423 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157423 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157423 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157511 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.157704 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.157527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.258730 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.258700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.258730 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.258735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.258979 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.258753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.258979 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.258777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.258979 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.258810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.258979 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:31.258862 2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 08 06:59:31.258979 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:31.258932 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls podName:1657ff88-3799-48ab-bb93-a30e7675978a nodeName:}" failed. No retries permitted until 2026-07-08 06:59:31.758909517 +0000 UTC m=+284.472558339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a") : secret "alertmanager-main-tls" not found Jul 08 06:59:31.258979 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.258969 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmc7d\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-kube-api-access-gmc7d\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259089 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-config-out\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-web-config\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259214 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.259290 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.260636 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.259986 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.260636 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.260005 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.260963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.260915 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.261762 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.261734 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.261870 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.261844 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.262479 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.262446 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.262479 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.262460 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.262642 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.262478 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-config-out\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.262773 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.262751 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.263307 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.263290 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-web-config\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.263702 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.263660 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.268848 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.268829 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmc7d\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-kube-api-access-gmc7d\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.763080 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.762993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.763569 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.763487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swwlb" event={"ID":"27623fbf-ed7a-4647-9200-68f9f3c4eed8","Type":"ContainerStarted","Data":"415ed5d6765c189c1616d84948e6687c9bb2b3e084f6d4c9e671231e6804334f"} Jul 08 06:59:31.765055 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.764984 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-zb79f" event={"ID":"9766fd0b-a971-4e8d-a126-01cb547a7f34","Type":"ContainerStarted","Data":"4a196b6e09ded2205e0acf61ee6ce2ca457cdd83e3bc69188bc29a4c5457f3a7"} Jul 08 06:59:31.766114 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.766067 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:31.970526 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:31.970507 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 06:59:32.129082 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:32.129052 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 06:59:32.131441 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:32.131413 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1657ff88_3799_48ab_bb93_a30e7675978a.slice/crio-7c4802df6a0a829bba253afc075132120afe043ec0f7d4dfc55a23529331026e WatchSource:0}: Error finding container 7c4802df6a0a829bba253afc075132120afe043ec0f7d4dfc55a23529331026e: Status 404 returned error can't find the container with id 7c4802df6a0a829bba253afc075132120afe043ec0f7d4dfc55a23529331026e Jul 08 06:59:32.770275 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:32.770234 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"7c4802df6a0a829bba253afc075132120afe043ec0f7d4dfc55a23529331026e"} Jul 08 06:59:32.772398 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:32.772323 2568 generic.go:356] "Generic (PLEG): container finished" podID="27623fbf-ed7a-4647-9200-68f9f3c4eed8" containerID="311953a97fe6aba0b714b161e021f88ff11fd8807538b0481b59cb8577285aeb" exitCode=0 Jul 08 06:59:32.772398 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:32.772354 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swwlb" event={"ID":"27623fbf-ed7a-4647-9200-68f9f3c4eed8","Type":"ContainerDied","Data":"311953a97fe6aba0b714b161e021f88ff11fd8807538b0481b59cb8577285aeb"} Jul 08 06:59:32.976102 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:32.975861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:59:32.978695 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:32.978666 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e26c3381-ae7d-4f03-a66f-163d012fbb99-metrics-tls\") pod \"dns-default-t5vjh\" (UID: \"e26c3381-ae7d-4f03-a66f-163d012fbb99\") " pod="openshift-dns/dns-default-t5vjh" Jul 08 06:59:33.055375 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.055101 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vxwlb\"" Jul 08 06:59:33.058902 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.058017 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-554b9948fc-sdpv2"] Jul 08 06:59:33.062781 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.062615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.063052 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.063034 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t5vjh" Jul 08 06:59:33.065214 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.065192 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 06:59:33.065457 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.065427 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 06:59:33.065719 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.065700 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 06:59:33.065944 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.065900 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-67v1mnvgo7dvf\"" Jul 08 06:59:33.066137 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.066121 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 06:59:33.066243 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.066169 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bhhsd\"" Jul 08 06:59:33.066381 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.066364 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 06:59:33.075201 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.075150 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-554b9948fc-sdpv2"] Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8ec52-0b94-4c87-ba8c-919f55575290-metrics-client-ca\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076804 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbpj8\" (UniqueName: \"kubernetes.io/projected/3ae8ec52-0b94-4c87-ba8c-919f55575290-kube-api-access-tbpj8\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-grpc-tls\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-tls\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.077849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.076953 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.177963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.177734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-tls\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.177963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.177797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.177963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.177863 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8ec52-0b94-4c87-ba8c-919f55575290-metrics-client-ca\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.177963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.177935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.177963 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.177964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbpj8\" (UniqueName: \"kubernetes.io/projected/3ae8ec52-0b94-4c87-ba8c-919f55575290-kube-api-access-tbpj8\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.178351 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.177992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-grpc-tls\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.179518 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.178775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae8ec52-0b94-4c87-ba8c-919f55575290-metrics-client-ca\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.179518 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.178831 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.179518 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.178864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.182321 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.182259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-tls\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.182907 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.182840 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.184072 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.184022 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.184189 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.184111 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.186705 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.186479 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-grpc-tls\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.186705 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.186667 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ae8ec52-0b94-4c87-ba8c-919f55575290-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.188759 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.188707 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbpj8\" (UniqueName: \"kubernetes.io/projected/3ae8ec52-0b94-4c87-ba8c-919f55575290-kube-api-access-tbpj8\") pod \"thanos-querier-554b9948fc-sdpv2\" (UID: \"3ae8ec52-0b94-4c87-ba8c-919f55575290\") " pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.380716 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.380629 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:33.404412 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.404337 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t5vjh"] Jul 08 06:59:33.408343 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:33.408314 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26c3381_ae7d_4f03_a66f_163d012fbb99.slice/crio-47afbf068cf6548f42185ec608f063dc9d1c5c54649ebedb63cb5d236cb45ba4 WatchSource:0}: Error finding container 47afbf068cf6548f42185ec608f063dc9d1c5c54649ebedb63cb5d236cb45ba4: Status 404 returned error can't find the container with id 47afbf068cf6548f42185ec608f063dc9d1c5c54649ebedb63cb5d236cb45ba4 Jul 08 06:59:33.540653 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.540574 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-554b9948fc-sdpv2"] Jul 08 06:59:33.544571 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:33.544544 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae8ec52_0b94_4c87_ba8c_919f55575290.slice/crio-971628df1c4cbbc05506f4f0c2fc804a7bf85d3f4de20d282d215f426ed77583 WatchSource:0}: Error finding container 971628df1c4cbbc05506f4f0c2fc804a7bf85d3f4de20d282d215f426ed77583: Status 404 returned error can't find the container with id 971628df1c4cbbc05506f4f0c2fc804a7bf85d3f4de20d282d215f426ed77583 Jul 08 06:59:33.777746 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.777706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t5vjh" event={"ID":"e26c3381-ae7d-4f03-a66f-163d012fbb99","Type":"ContainerStarted","Data":"47afbf068cf6548f42185ec608f063dc9d1c5c54649ebedb63cb5d236cb45ba4"} Jul 08 06:59:33.779795 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.779730 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef" exitCode=0 Jul 08 06:59:33.779952 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.779808 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef"} Jul 08 06:59:33.781128 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.781098 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"971628df1c4cbbc05506f4f0c2fc804a7bf85d3f4de20d282d215f426ed77583"} Jul 08 06:59:33.783797 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.783773 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swwlb" event={"ID":"27623fbf-ed7a-4647-9200-68f9f3c4eed8","Type":"ContainerStarted","Data":"14d78885189238a8148e013b93e6cc96d4baa7d01b847e47d6d118fb7fe01049"} Jul 08 06:59:33.783911 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.783820 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swwlb" event={"ID":"27623fbf-ed7a-4647-9200-68f9f3c4eed8","Type":"ContainerStarted","Data":"9bb5b4cb7300bfaa865abd3cee2931f687eea46c51a2a29268fc85b0d6ec7f17"} Jul 08 06:59:33.843205 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:33.843130 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-swwlb" podStartSLOduration=3.809954083 podStartE2EDuration="4.843111372s" totalImagesPullingTime="956.688503ms" totalInitContainerRuntime="76.468786ms" isStatefulPod=true podCreationTimestamp="2026-07-08 06:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:33.842874724 +0000 UTC m=+286.556523562" watchObservedRunningTime="2026-07-08 06:59:33.843111372 +0000 UTC m=+286.556760209" Jul 08 06:59:34.398236 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.396773 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg"] Jul 08 06:59:34.400390 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.400367 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.406020 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.405121 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 06:59:34.406020 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.405363 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-er9kkkvqnkf36\"" Jul 08 06:59:34.406020 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.405559 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 06:59:34.406020 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.405617 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 06:59:34.406020 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.405820 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 06:59:34.406396 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.406113 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5cgzw\"" Jul 08 06:59:34.414075 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.411501 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg"] Jul 08 06:59:34.490915 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.490849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-secret-metrics-server-client-certs\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.491097 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.490968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-secret-metrics-server-tls\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.491097 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.491008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3254d724-4d1d-4223-a780-9817b26f871c-audit-log\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.491097 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.491061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-client-ca-bundle\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.491097 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.491086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp857\" (UniqueName: \"kubernetes.io/projected/3254d724-4d1d-4223-a780-9817b26f871c-kube-api-access-qp857\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.491333 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.491137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3254d724-4d1d-4223-a780-9817b26f871c-metrics-server-audit-profiles\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.491333 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.491230 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3254d724-4d1d-4223-a780-9817b26f871c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592531 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592487 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3254d724-4d1d-4223-a780-9817b26f871c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-secret-metrics-server-client-certs\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-secret-metrics-server-tls\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3254d724-4d1d-4223-a780-9817b26f871c-audit-log\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-client-ca-bundle\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592712 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp857\" (UniqueName: \"kubernetes.io/projected/3254d724-4d1d-4223-a780-9817b26f871c-kube-api-access-qp857\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.592980 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.592752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3254d724-4d1d-4223-a780-9817b26f871c-metrics-server-audit-profiles\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.593761 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.593586 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/3254d724-4d1d-4223-a780-9817b26f871c-audit-log\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.593935 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.593900 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/3254d724-4d1d-4223-a780-9817b26f871c-metrics-server-audit-profiles\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.594008 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.593972 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3254d724-4d1d-4223-a780-9817b26f871c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.597562 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.597539 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-secret-metrics-server-client-certs\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.598602 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.598561 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-secret-metrics-server-tls\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.599400 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.599362 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3254d724-4d1d-4223-a780-9817b26f871c-client-ca-bundle\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.606002 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.605960 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp857\" (UniqueName: \"kubernetes.io/projected/3254d724-4d1d-4223-a780-9817b26f871c-kube-api-access-qp857\") pod \"metrics-server-7bfccb7cc4-ntfxg\" (UID: \"3254d724-4d1d-4223-a780-9817b26f871c\") " pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.717038 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.716976 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:34.906557 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:34.906513 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg"] Jul 08 06:59:34.911713 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:34.911683 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3254d724_4d1d_4223_a780_9817b26f871c.slice/crio-d2512bcad421502dee659245a40d42de8f7a4427604538f28248a25593fea8fd WatchSource:0}: Error finding container d2512bcad421502dee659245a40d42de8f7a4427604538f28248a25593fea8fd: Status 404 returned error can't find the container with id d2512bcad421502dee659245a40d42de8f7a4427604538f28248a25593fea8fd Jul 08 06:59:35.176433 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.176319 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5865b876d7-z7kjw"] Jul 08 06:59:35.180643 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.180617 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.184305 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.184280 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v758n\"" Jul 08 06:59:35.184441 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.184382 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 06:59:35.184499 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.184476 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 06:59:35.184598 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.184579 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 06:59:35.184666 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.184608 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 06:59:35.184719 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.184666 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 06:59:35.190027 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.189967 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5865b876d7-z7kjw"] Jul 08 06:59:35.203305 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.203264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-oauth-config\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.203457 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.203361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-config\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.203457 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.203438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-oauth-serving-cert\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.203571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.203475 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbzlz\" (UniqueName: \"kubernetes.io/projected/3c422ee2-27bd-4c44-b722-a5813bc6eda0-kube-api-access-nbzlz\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.203571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.203512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-serving-cert\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.203571 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.203535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-service-ca\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.304230 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.303982 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbzlz\" (UniqueName: \"kubernetes.io/projected/3c422ee2-27bd-4c44-b722-a5813bc6eda0-kube-api-access-nbzlz\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.304230 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.304033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-serving-cert\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.304230 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.304060 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-service-ca\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.304230 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.304104 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-oauth-config\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.304230 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.304217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-config\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.304611 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.304246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-oauth-serving-cert\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.305106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.305057 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-oauth-serving-cert\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.305356 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.305332 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-service-ca\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.305891 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.305837 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-config\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.307480 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.307460 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-serving-cert\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.307919 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.307891 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-oauth-config\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.318737 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.318692 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbzlz\" (UniqueName: \"kubernetes.io/projected/3c422ee2-27bd-4c44-b722-a5813bc6eda0-kube-api-access-nbzlz\") pod \"console-5865b876d7-z7kjw\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.494873 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.494842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:35.815410 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:35.815323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" event={"ID":"3254d724-4d1d-4223-a780-9817b26f871c","Type":"ContainerStarted","Data":"d2512bcad421502dee659245a40d42de8f7a4427604538f28248a25593fea8fd"} Jul 08 06:59:36.915167 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:36.915118 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5865b876d7-z7kjw"] Jul 08 06:59:37.703534 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.703510 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56986c54d8-scx9h" Jul 08 06:59:37.842543 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.842459 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134"} Jul 08 06:59:37.842774 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.842751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61"} Jul 08 06:59:37.842849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.842802 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4"} Jul 08 06:59:37.842849 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.842845 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217"} Jul 08 06:59:37.842960 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.842884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a"} Jul 08 06:59:37.845665 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.845480 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"2c3d933f086c62b2dd728c4a17419ddd6053a31e06f54e87972d79ea63a7615c"} Jul 08 06:59:37.845665 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.845555 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"aeab8a641787028ff134e964b9cca0363739b4b1ec98fb79d4082987469670ed"} Jul 08 06:59:37.845665 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.845601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"e40c813ff9323d7c3638acd3976229fb6fcbcb02434fb190e13ea6812c0eab2d"} Jul 08 06:59:37.847530 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.847505 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5865b876d7-z7kjw" event={"ID":"3c422ee2-27bd-4c44-b722-a5813bc6eda0","Type":"ContainerStarted","Data":"eca3784713e6d8b429532795824bbc5ec2c1aa9f90ad64ef443a48b2c7374d1c"} Jul 08 06:59:37.849515 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.849494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t5vjh" event={"ID":"e26c3381-ae7d-4f03-a66f-163d012fbb99","Type":"ContainerStarted","Data":"8a6a7678b206d3762ef2338ea72ea1605a2a4ade0d65913534aef7caf0cb7d55"} Jul 08 06:59:37.849661 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.849643 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t5vjh" event={"ID":"e26c3381-ae7d-4f03-a66f-163d012fbb99","Type":"ContainerStarted","Data":"ba83b958f86091a80c6fd8fd8ab16f8baf78157646d9e017403f334646d4a321"} Jul 08 06:59:37.849781 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.849768 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t5vjh" Jul 08 06:59:37.851784 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.851746 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" event={"ID":"3254d724-4d1d-4223-a780-9817b26f871c","Type":"ContainerStarted","Data":"34e43fe989164ba37a507043e716d0d754f70c54b866539862d2ebb24db6eb4d"} Jul 08 06:59:37.891731 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.891674 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t5vjh" podStartSLOduration=251.065333641 podStartE2EDuration="4m14.891655034s" totalImagesPullingTime="3.826321393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:37.890752213 +0000 UTC m=+290.604401050" watchObservedRunningTime="2026-07-08 06:59:37.891655034 +0000 UTC m=+290.605303847" Jul 08 06:59:37.927926 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:37.927831 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" podStartSLOduration=1.5529174650000002 podStartE2EDuration="3.9278097s" totalImagesPullingTime="2.374892235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:37.926603249 +0000 UTC m=+290.640252077" watchObservedRunningTime="2026-07-08 06:59:37.9278097 +0000 UTC m=+290.641458540" Jul 08 06:59:38.862249 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:38.861413 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerStarted","Data":"b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6"} Jul 08 06:59:38.864923 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:38.864890 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"19331611a49f77e4d870b0babe45305d2c18c63d42658ad17ff660068ec04f71"} Jul 08 06:59:38.865106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:38.864968 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"b64b8ed3092990b4b122f9289a919c1ae2ef5e0877a0aac5a830f51711e1071e"} Jul 08 06:59:38.865106 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:38.865012 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" event={"ID":"3ae8ec52-0b94-4c87-ba8c-919f55575290","Type":"ContainerStarted","Data":"bba5a9089c609856d94d1239f95ef6089f8cb2be580f03db71cc9dbbc5ee926b"} Jul 08 06:59:38.902201 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:38.902002 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.910179199 podStartE2EDuration="7.901980228s" totalImagesPullingTime="4.964566s" totalInitContainerRuntime="27.235029ms" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:38.900058254 +0000 UTC m=+291.613707091" watchObservedRunningTime="2026-07-08 06:59:38.901980228 +0000 UTC m=+291.615629066" Jul 08 06:59:38.929220 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:38.928347 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" podStartSLOduration=1.753611388 podStartE2EDuration="5.928325352s" totalImagesPullingTime="4.174713964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:38.926949852 +0000 UTC m=+291.640598689" watchObservedRunningTime="2026-07-08 06:59:38.928325352 +0000 UTC m=+291.641974190" Jul 08 06:59:39.871056 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:39.871023 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:40.876934 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:40.876837 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5865b876d7-z7kjw" event={"ID":"3c422ee2-27bd-4c44-b722-a5813bc6eda0","Type":"ContainerStarted","Data":"2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1"} Jul 08 06:59:40.894532 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:40.894483 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5865b876d7-z7kjw" podStartSLOduration=2.6620268449999998 podStartE2EDuration="5.894468164s" totalImagesPullingTime="3.232441319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:40.893570815 +0000 UTC m=+293.607219657" watchObservedRunningTime="2026-07-08 06:59:40.894468164 +0000 UTC m=+293.608116999" Jul 08 06:59:44.164109 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.164071 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74cf656bd-8fxns"] Jul 08 06:59:44.168485 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.168465 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.178795 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.178667 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 06:59:44.179123 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.179096 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cf656bd-8fxns"] Jul 08 06:59:44.295459 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkbn5\" (UniqueName: \"kubernetes.io/projected/bcf3a159-50c9-46c3-8536-9499d3dd6e07-kube-api-access-bkbn5\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.295631 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295605 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-serving-cert\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.295688 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-trusted-ca-bundle\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.295758 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295728 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-service-ca\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.295806 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-oauth-config\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.295853 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-config\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.295853 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.295838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-oauth-serving-cert\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397373 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-serving-cert\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397550 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-trusted-ca-bundle\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397550 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-service-ca\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397550 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397543 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-oauth-config\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397694 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-config\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397694 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-oauth-serving-cert\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.397694 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.397644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbn5\" (UniqueName: \"kubernetes.io/projected/bcf3a159-50c9-46c3-8536-9499d3dd6e07-kube-api-access-bkbn5\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.398460 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.398408 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-config\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.398460 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.398422 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-service-ca\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.398610 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.398532 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-oauth-serving-cert\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.398880 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.398855 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-trusted-ca-bundle\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.400341 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.400317 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-oauth-config\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.400449 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.400317 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-serving-cert\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.406641 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.406616 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkbn5\" (UniqueName: \"kubernetes.io/projected/bcf3a159-50c9-46c3-8536-9499d3dd6e07-kube-api-access-bkbn5\") pod \"console-74cf656bd-8fxns\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.480753 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:44.480703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:44.821357 ip-10-0-138-98 kubenswrapper[2568]: E0708 06:59:44.821259 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kmdj4" podUID="350c83c0-715e-4925-8f3e-27ff51dee9b1" Jul 08 06:59:45.495339 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:45.495311 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:45.495828 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:45.495362 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:45.500602 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:45.500577 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:45.884592 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:45.884520 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-554b9948fc-sdpv2" Jul 08 06:59:45.904950 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:45.904927 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 06:59:47.868243 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:47.868065 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t5vjh" Jul 08 06:59:49.193558 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.193532 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 06:59:49.194855 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.194830 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 06:59:49.209932 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.209907 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 06:59:49.298666 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.298627 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cf656bd-8fxns"] Jul 08 06:59:49.300781 ip-10-0-138-98 kubenswrapper[2568]: W0708 06:59:49.300756 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf3a159_50c9_46c3_8536_9499d3dd6e07.slice/crio-9f0921f14f3a06bc0f5a04759bc526be3200805b3f49133796664596ea99eea0 WatchSource:0}: Error finding container 9f0921f14f3a06bc0f5a04759bc526be3200805b3f49133796664596ea99eea0: Status 404 returned error can't find the container with id 9f0921f14f3a06bc0f5a04759bc526be3200805b3f49133796664596ea99eea0 Jul 08 06:59:49.918941 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.918837 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-zb79f" event={"ID":"9766fd0b-a971-4e8d-a126-01cb547a7f34","Type":"ContainerStarted","Data":"46c66f369a4cad2b8ec3ca66a1dbc2963d70c94d37234da7d28e83e1e497ec91"} Jul 08 06:59:49.918941 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.918906 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:49.920463 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.920434 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf656bd-8fxns" event={"ID":"bcf3a159-50c9-46c3-8536-9499d3dd6e07","Type":"ContainerStarted","Data":"5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c"} Jul 08 06:59:49.920463 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.920426 2568 patch_prober.go:28] interesting pod/downloads-685bfb5f64-zb79f container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.15:8080/\": dial tcp 10.134.0.15:8080: connect: connection refused" start-of-body= Jul 08 06:59:49.920604 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.920476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf656bd-8fxns" event={"ID":"bcf3a159-50c9-46c3-8536-9499d3dd6e07","Type":"ContainerStarted","Data":"9f0921f14f3a06bc0f5a04759bc526be3200805b3f49133796664596ea99eea0"} Jul 08 06:59:49.920604 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.920506 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-zb79f" podUID="9766fd0b-a971-4e8d-a126-01cb547a7f34" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.15:8080/\": dial tcp 10.134.0.15:8080: connect: connection refused" Jul 08 06:59:49.936635 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.936585 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-zb79f" podStartSLOduration=1.518583892 podStartE2EDuration="19.936570274s" totalImagesPullingTime="18.417986382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:49.935612026 +0000 UTC m=+302.649260862" watchObservedRunningTime="2026-07-08 06:59:49.936570274 +0000 UTC m=+302.650219109" Jul 08 06:59:49.952388 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:49.952342 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74cf656bd-8fxns" podStartSLOduration=5.952327317 podStartE2EDuration="5.952327317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:59:49.950988078 +0000 UTC m=+302.664636914" watchObservedRunningTime="2026-07-08 06:59:49.952327317 +0000 UTC m=+302.665976153" Jul 08 06:59:50.941276 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:50.941241 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-zb79f" Jul 08 06:59:54.481143 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:54.481100 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:54.481701 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:54.481219 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:54.486675 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:54.486650 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:54.717833 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:54.717787 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:54.718005 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:54.717860 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 06:59:54.954207 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:54.954151 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 06:59:55.000357 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:55.000317 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5865b876d7-z7kjw"] Jul 08 06:59:55.819275 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:55.819239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 06:59:59.972313 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:59.972279 2568 generic.go:356] "Generic (PLEG): container finished" podID="018bbf07-db34-465e-a565-5cb929e86098" containerID="2b7233648d686106ad99c7aa0455b81d03992271af8afa098bc6dfdd09c26284" exitCode=0 Jul 08 06:59:59.972884 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:59.972352 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" event={"ID":"018bbf07-db34-465e-a565-5cb929e86098","Type":"ContainerDied","Data":"2b7233648d686106ad99c7aa0455b81d03992271af8afa098bc6dfdd09c26284"} Jul 08 06:59:59.972884 ip-10-0-138-98 kubenswrapper[2568]: I0708 06:59:59.972762 2568 scope.go:117] "RemoveContainer" containerID="2b7233648d686106ad99c7aa0455b81d03992271af8afa098bc6dfdd09c26284" Jul 08 07:00:00.978143 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:00.978115 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gtqth" event={"ID":"018bbf07-db34-465e-a565-5cb929e86098","Type":"ContainerStarted","Data":"2485e659a789bbb5876c7ac459e444963b1f6cad3089e211b5fd6041034ab032"} Jul 08 07:00:14.723601 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:14.723566 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 07:00:14.727386 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:14.727363 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7bfccb7cc4-ntfxg" Jul 08 07:00:20.028577 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.028507 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5865b876d7-z7kjw" podUID="3c422ee2-27bd-4c44-b722-a5813bc6eda0" containerName="console" containerID="cri-o://2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1" gracePeriod=15 Jul 08 07:00:20.323261 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.323233 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 07:00:20.433192 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433141 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-service-ca\") pod \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " Jul 08 07:00:20.433380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433205 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-serving-cert\") pod \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " Jul 08 07:00:20.433380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433232 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-oauth-serving-cert\") pod \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " Jul 08 07:00:20.433380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433272 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbzlz\" (UniqueName: \"kubernetes.io/projected/3c422ee2-27bd-4c44-b722-a5813bc6eda0-kube-api-access-nbzlz\") pod \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " Jul 08 07:00:20.433380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433316 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-config\") pod \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " Jul 08 07:00:20.433380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433340 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-oauth-config\") pod \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\" (UID: \"3c422ee2-27bd-4c44-b722-a5813bc6eda0\") " Jul 08 07:00:20.433724 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433685 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-service-ca" (OuterVolumeSpecName: "service-ca") pod "3c422ee2-27bd-4c44-b722-a5813bc6eda0" (UID: "3c422ee2-27bd-4c44-b722-a5813bc6eda0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:00:20.433724 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433710 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3c422ee2-27bd-4c44-b722-a5813bc6eda0" (UID: "3c422ee2-27bd-4c44-b722-a5813bc6eda0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:00:20.433991 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.433932 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-config" (OuterVolumeSpecName: "console-config") pod "3c422ee2-27bd-4c44-b722-a5813bc6eda0" (UID: "3c422ee2-27bd-4c44-b722-a5813bc6eda0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:00:20.435727 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.435705 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3c422ee2-27bd-4c44-b722-a5813bc6eda0" (UID: "3c422ee2-27bd-4c44-b722-a5813bc6eda0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:20.435931 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.435913 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3c422ee2-27bd-4c44-b722-a5813bc6eda0" (UID: "3c422ee2-27bd-4c44-b722-a5813bc6eda0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:20.435993 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.435925 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c422ee2-27bd-4c44-b722-a5813bc6eda0-kube-api-access-nbzlz" (OuterVolumeSpecName: "kube-api-access-nbzlz") pod "3c422ee2-27bd-4c44-b722-a5813bc6eda0" (UID: "3c422ee2-27bd-4c44-b722-a5813bc6eda0"). InnerVolumeSpecName "kube-api-access-nbzlz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:00:20.534136 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.534103 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbzlz\" (UniqueName: \"kubernetes.io/projected/3c422ee2-27bd-4c44-b722-a5813bc6eda0-kube-api-access-nbzlz\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:20.534136 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.534130 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:20.534136 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.534139 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-oauth-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:20.534136 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.534147 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-service-ca\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:20.534408 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.534180 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c422ee2-27bd-4c44-b722-a5813bc6eda0-console-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:20.534408 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:20.534189 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c422ee2-27bd-4c44-b722-a5813bc6eda0-oauth-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:21.064838 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.064801 2568 generic.go:356] "Generic (PLEG): container finished" podID="3c422ee2-27bd-4c44-b722-a5813bc6eda0" containerID="2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1" exitCode=0 Jul 08 07:00:21.065235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.064851 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5865b876d7-z7kjw" Jul 08 07:00:21.065235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.064886 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5865b876d7-z7kjw" event={"ID":"3c422ee2-27bd-4c44-b722-a5813bc6eda0","Type":"ContainerDied","Data":"2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1"} Jul 08 07:00:21.065235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.064934 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5865b876d7-z7kjw" event={"ID":"3c422ee2-27bd-4c44-b722-a5813bc6eda0","Type":"ContainerDied","Data":"eca3784713e6d8b429532795824bbc5ec2c1aa9f90ad64ef443a48b2c7374d1c"} Jul 08 07:00:21.065235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.064957 2568 scope.go:117] "RemoveContainer" containerID="2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1" Jul 08 07:00:21.073528 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.073508 2568 scope.go:117] "RemoveContainer" containerID="2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1" Jul 08 07:00:21.073778 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:21.073762 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1\": container with ID starting with 2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1 not found: ID does not exist" containerID="2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1" Jul 08 07:00:21.073822 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.073785 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1"} err="failed to get container status \"2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1\": rpc error: code = NotFound desc = could not find container \"2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1\": container with ID starting with 2a136c20734634ea683b11b94a65a606763572520d6e684dc97b0f1f2b4e01c1 not found: ID does not exist" Jul 08 07:00:21.085493 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.085387 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5865b876d7-z7kjw"] Jul 08 07:00:21.087218 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.087200 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5865b876d7-z7kjw"] Jul 08 07:00:21.824940 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:21.824908 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c422ee2-27bd-4c44-b722-a5813bc6eda0" path="/var/lib/kubelet/pods/3c422ee2-27bd-4c44-b722-a5813bc6eda0/volumes" Jul 08 07:00:50.331696 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.331665 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 07:00:50.332176 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.331968 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="alertmanager" containerID="cri-o://3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a" gracePeriod=120 Jul 08 07:00:50.332176 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.332012 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-metric" containerID="cri-o://dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134" gracePeriod=120 Jul 08 07:00:50.332176 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.332042 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-web" containerID="cri-o://fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4" gracePeriod=120 Jul 08 07:00:50.332176 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.332080 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="config-reloader" containerID="cri-o://c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217" gracePeriod=120 Jul 08 07:00:50.332176 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.332082 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy" containerID="cri-o://bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61" gracePeriod=120 Jul 08 07:00:50.332176 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:50.332105 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="prom-label-proxy" containerID="cri-o://b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6" gracePeriod=120 Jul 08 07:00:51.177836 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177803 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6" exitCode=0 Jul 08 07:00:51.177836 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177827 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134" exitCode=0 Jul 08 07:00:51.177836 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177833 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61" exitCode=0 Jul 08 07:00:51.177836 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177838 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217" exitCode=0 Jul 08 07:00:51.177836 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177844 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a" exitCode=0 Jul 08 07:00:51.178119 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177872 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6"} Jul 08 07:00:51.178119 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177929 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134"} Jul 08 07:00:51.178119 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61"} Jul 08 07:00:51.178119 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.177979 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217"} Jul 08 07:00:51.178119 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.178002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a"} Jul 08 07:00:51.578629 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.578607 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:51.606491 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606465 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.606620 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606501 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-tls-assets\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.606620 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606534 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.606753 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606729 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-web-config\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.606825 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606773 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-config-volume\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.606825 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606800 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-trusted-ca-bundle\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.607068 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606825 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-cluster-tls-config\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.607068 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606882 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-config-out\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.607068 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606905 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-web\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.607068 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.606968 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-main-db\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.608315 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.607003 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-metrics-client-ca\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.608315 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.607714 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmc7d\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-kube-api-access-gmc7d\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.608315 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.607746 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls\") pod \"1657ff88-3799-48ab-bb93-a30e7675978a\" (UID: \"1657ff88-3799-48ab-bb93-a30e7675978a\") " Jul 08 07:00:51.608315 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.608003 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:00:51.608315 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.608091 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.609208 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.608829 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:00:51.609208 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.609184 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 07:00:51.610009 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.609982 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:00:51.610655 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.610609 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.610655 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.610616 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-config-out" (OuterVolumeSpecName: "config-out") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 07:00:51.611003 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.610979 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.611463 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.611428 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.611555 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.611479 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-config-volume" (OuterVolumeSpecName: "config-volume") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.612276 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.612237 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.613349 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.613311 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-kube-api-access-gmc7d" (OuterVolumeSpecName: "kube-api-access-gmc7d") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "kube-api-access-gmc7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:00:51.617307 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.617226 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.624455 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.624432 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-web-config" (OuterVolumeSpecName: "web-config") pod "1657ff88-3799-48ab-bb93-a30e7675978a" (UID: "1657ff88-3799-48ab-bb93-a30e7675978a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:00:51.709121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709043 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-web-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709068 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-config-volume\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709079 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-cluster-tls-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709106 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-config-out\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709115 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709126 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1657ff88-3799-48ab-bb93-a30e7675978a-alertmanager-main-db\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709478 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709136 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1657ff88-3799-48ab-bb93-a30e7675978a-metrics-client-ca\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709478 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709144 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmc7d\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-kube-api-access-gmc7d\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709478 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709173 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-main-tls\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709478 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709182 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709478 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709190 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1657ff88-3799-48ab-bb93-a30e7675978a-tls-assets\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:51.709478 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:51.709200 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1657ff88-3799-48ab-bb93-a30e7675978a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:00:52.184380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.184292 2568 generic.go:356] "Generic (PLEG): container finished" podID="1657ff88-3799-48ab-bb93-a30e7675978a" containerID="fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4" exitCode=0 Jul 08 07:00:52.184380 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.184353 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4"} Jul 08 07:00:52.184546 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.184382 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.184546 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.184422 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1657ff88-3799-48ab-bb93-a30e7675978a","Type":"ContainerDied","Data":"7c4802df6a0a829bba253afc075132120afe043ec0f7d4dfc55a23529331026e"} Jul 08 07:00:52.184546 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.184460 2568 scope.go:117] "RemoveContainer" containerID="b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6" Jul 08 07:00:52.193526 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.193507 2568 scope.go:117] "RemoveContainer" containerID="dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134" Jul 08 07:00:52.200473 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.200449 2568 scope.go:117] "RemoveContainer" containerID="bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61" Jul 08 07:00:52.205985 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.205964 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 07:00:52.207205 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.207189 2568 scope.go:117] "RemoveContainer" containerID="fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4" Jul 08 07:00:52.210939 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.210920 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 07:00:52.213984 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.213968 2568 scope.go:117] "RemoveContainer" containerID="c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217" Jul 08 07:00:52.220730 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.220711 2568 scope.go:117] "RemoveContainer" containerID="3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a" Jul 08 07:00:52.227403 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.227382 2568 scope.go:117] "RemoveContainer" containerID="35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef" Jul 08 07:00:52.234008 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.233990 2568 scope.go:117] "RemoveContainer" containerID="b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6" Jul 08 07:00:52.234374 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.234351 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6\": container with ID starting with b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6 not found: ID does not exist" containerID="b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6" Jul 08 07:00:52.234472 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.234387 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6"} err="failed to get container status \"b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6\": rpc error: code = NotFound desc = could not find container \"b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6\": container with ID starting with b6e42847e8c5a89367cedd82235c7d2f16481c9d0a146ce02cc13e8880d653d6 not found: ID does not exist" Jul 08 07:00:52.234472 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.234414 2568 scope.go:117] "RemoveContainer" containerID="dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134" Jul 08 07:00:52.234808 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.234785 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134\": container with ID starting with dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134 not found: ID does not exist" containerID="dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134" Jul 08 07:00:52.234870 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.234815 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134"} err="failed to get container status \"dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134\": rpc error: code = NotFound desc = could not find container \"dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134\": container with ID starting with dc7446717f85b160e8e090cd86a257582797c4a92b78cb7146de381a62ab3134 not found: ID does not exist" Jul 08 07:00:52.234870 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.234838 2568 scope.go:117] "RemoveContainer" containerID="bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61" Jul 08 07:00:52.235107 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.235070 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61\": container with ID starting with bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61 not found: ID does not exist" containerID="bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61" Jul 08 07:00:52.235181 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235116 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61"} err="failed to get container status \"bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61\": rpc error: code = NotFound desc = could not find container \"bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61\": container with ID starting with bc7a595bf066f6b5e31fa8b1fa551f6c771147cf16058888e5eb62ba29e26f61 not found: ID does not exist" Jul 08 07:00:52.235181 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235136 2568 scope.go:117] "RemoveContainer" containerID="fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4" Jul 08 07:00:52.235420 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.235402 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4\": container with ID starting with fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4 not found: ID does not exist" containerID="fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4" Jul 08 07:00:52.235477 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235425 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4"} err="failed to get container status \"fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4\": rpc error: code = NotFound desc = could not find container \"fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4\": container with ID starting with fa46516e109897476fac199160013b016b9256cc6145e86eb3c5a8e216c6a7d4 not found: ID does not exist" Jul 08 07:00:52.235477 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235441 2568 scope.go:117] "RemoveContainer" containerID="c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217" Jul 08 07:00:52.235633 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.235617 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217\": container with ID starting with c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217 not found: ID does not exist" containerID="c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217" Jul 08 07:00:52.235672 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235636 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217"} err="failed to get container status \"c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217\": rpc error: code = NotFound desc = could not find container \"c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217\": container with ID starting with c03cd21c39a1c59f51c08f91fcbdbdc9ef9e1f7c5b49e34cc14794571091a217 not found: ID does not exist" Jul 08 07:00:52.235672 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235648 2568 scope.go:117] "RemoveContainer" containerID="3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a" Jul 08 07:00:52.235813 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.235795 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a\": container with ID starting with 3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a not found: ID does not exist" containerID="3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a" Jul 08 07:00:52.235878 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235831 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a"} err="failed to get container status \"3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a\": rpc error: code = NotFound desc = could not find container \"3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a\": container with ID starting with 3f4030b8af30ea8018cf6c01e4447457f7ed6068b6207d072717c402328c958a not found: ID does not exist" Jul 08 07:00:52.235878 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235846 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 07:00:52.235968 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.235853 2568 scope.go:117] "RemoveContainer" containerID="35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef" Jul 08 07:00:52.236191 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236137 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy" Jul 08 07:00:52.236191 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:00:52.236148 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef\": container with ID starting with 35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef not found: ID does not exist" containerID="35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef" Jul 08 07:00:52.236191 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236182 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef"} err="failed to get container status \"35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef\": rpc error: code = NotFound desc = could not find container \"35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef\": container with ID starting with 35dc036ec0446543eed250ca475430e730d72b4d7aa6245c60c0df7578b972ef not found: ID does not exist" Jul 08 07:00:52.236191 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236170 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236207 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="prom-label-proxy" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236220 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="prom-label-proxy" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236230 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-metric" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236236 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-metric" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236247 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="init-config-reloader" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236253 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="init-config-reloader" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236268 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="alertmanager" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236274 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="alertmanager" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236280 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c422ee2-27bd-4c44-b722-a5813bc6eda0" containerName="console" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236285 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c422ee2-27bd-4c44-b722-a5813bc6eda0" containerName="console" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236294 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-web" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236299 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-web" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236306 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="config-reloader" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236311 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="config-reloader" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236372 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="prom-label-proxy" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236380 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-web" Jul 08 07:00:52.236379 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236386 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="alertmanager" Jul 08 07:00:52.236852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236392 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="config-reloader" Jul 08 07:00:52.236852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236398 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy" Jul 08 07:00:52.236852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236402 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c422ee2-27bd-4c44-b722-a5813bc6eda0" containerName="console" Jul 08 07:00:52.236852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.236409 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" containerName="kube-rbac-proxy-metric" Jul 08 07:00:52.239584 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.239568 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.242033 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242018 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 07:00:52.242332 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242310 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 07:00:52.242419 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242313 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fs6cf\"" Jul 08 07:00:52.242419 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242334 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 07:00:52.242561 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242548 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 07:00:52.242607 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242589 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 07:00:52.242607 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242596 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 07:00:52.242882 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242868 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 07:00:52.242933 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.242870 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 07:00:52.247505 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.247484 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 07:00:52.256994 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.256973 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 07:00:52.315260 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315260 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315258 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0645e724-4f52-4ff9-ad2c-5c9232a231b8-config-out\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0645e724-4f52-4ff9-ad2c-5c9232a231b8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315320 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0645e724-4f52-4ff9-ad2c-5c9232a231b8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0645e724-4f52-4ff9-ad2c-5c9232a231b8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315714 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315714 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315563 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-web-config\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315714 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0645e724-4f52-4ff9-ad2c-5c9232a231b8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315714 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315649 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-config-volume\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.315714 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.315667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r52bh\" (UniqueName: \"kubernetes.io/projected/0645e724-4f52-4ff9-ad2c-5c9232a231b8-kube-api-access-r52bh\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416229 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416229 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416233 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416456 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0645e724-4f52-4ff9-ad2c-5c9232a231b8-config-out\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416456 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416287 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0645e724-4f52-4ff9-ad2c-5c9232a231b8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416456 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0645e724-4f52-4ff9-ad2c-5c9232a231b8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416456 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416456 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416361 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416689 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0645e724-4f52-4ff9-ad2c-5c9232a231b8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416689 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416793 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-web-config\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416793 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416709 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0645e724-4f52-4ff9-ad2c-5c9232a231b8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416793 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0645e724-4f52-4ff9-ad2c-5c9232a231b8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416933 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416795 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-config-volume\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.416933 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.416820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r52bh\" (UniqueName: \"kubernetes.io/projected/0645e724-4f52-4ff9-ad2c-5c9232a231b8-kube-api-access-r52bh\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.417398 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.417369 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0645e724-4f52-4ff9-ad2c-5c9232a231b8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.418305 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.418277 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0645e724-4f52-4ff9-ad2c-5c9232a231b8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.419563 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.419536 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.419661 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.419615 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.419816 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.419776 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0645e724-4f52-4ff9-ad2c-5c9232a231b8-config-out\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.420058 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.420036 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.420125 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.420039 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.420201 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.420143 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.420279 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.420263 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-web-config\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.420528 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.420513 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0645e724-4f52-4ff9-ad2c-5c9232a231b8-config-volume\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.421017 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.421004 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0645e724-4f52-4ff9-ad2c-5c9232a231b8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.424850 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.424826 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r52bh\" (UniqueName: \"kubernetes.io/projected/0645e724-4f52-4ff9-ad2c-5c9232a231b8-kube-api-access-r52bh\") pod \"alertmanager-main-0\" (UID: \"0645e724-4f52-4ff9-ad2c-5c9232a231b8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.548815 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.548779 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 07:00:52.683443 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:52.683420 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 07:00:52.685821 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:00:52.685796 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0645e724_4f52_4ff9_ad2c_5c9232a231b8.slice/crio-aa0c4bf87361c92ef433911d853893049b67c2e6277e04336520df1a275b3d4b WatchSource:0}: Error finding container aa0c4bf87361c92ef433911d853893049b67c2e6277e04336520df1a275b3d4b: Status 404 returned error can't find the container with id aa0c4bf87361c92ef433911d853893049b67c2e6277e04336520df1a275b3d4b Jul 08 07:00:53.190633 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:53.190599 2568 generic.go:356] "Generic (PLEG): container finished" podID="0645e724-4f52-4ff9-ad2c-5c9232a231b8" containerID="a745a91a88ba16ed4d76755402c083c7644cb179715d9b25a3ef1890b1442843" exitCode=0 Jul 08 07:00:53.190807 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:53.190649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerDied","Data":"a745a91a88ba16ed4d76755402c083c7644cb179715d9b25a3ef1890b1442843"} Jul 08 07:00:53.190807 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:53.190748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"aa0c4bf87361c92ef433911d853893049b67c2e6277e04336520df1a275b3d4b"} Jul 08 07:00:53.825201 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:53.825108 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1657ff88-3799-48ab-bb93-a30e7675978a" path="/var/lib/kubelet/pods/1657ff88-3799-48ab-bb93-a30e7675978a/volumes" Jul 08 07:00:54.197755 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.197724 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"e8ed61f98428060c70670593b45be9bb1fa7106372a65d252d6a35de3690118e"} Jul 08 07:00:54.197954 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.197797 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"5caa26c6cb5a139435d85c7804ae26bd81d39b7d8ba89c2abc01917539215d02"} Jul 08 07:00:54.197954 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.197837 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"0666f08a4b400961780510c0ca020e114ce6956f621fbc458ceda4c32a61df15"} Jul 08 07:00:54.197954 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.197870 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"6a0aa7ed7e34f632c1ab74792907311ec1ed0f5f8162a65329b2cfab1af3d282"} Jul 08 07:00:54.197954 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.197904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"2854d45017b5c3ca682bc3dd5a53d458cd3f0482a504d8000faa0341064f14fe"} Jul 08 07:00:54.197954 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.197944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0645e724-4f52-4ff9-ad2c-5c9232a231b8","Type":"ContainerStarted","Data":"2e4b312478295ff6dc2bf561d5f45828b5c65743812af21367a78cdeb23c8061"} Jul 08 07:00:54.226134 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.226078 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.207308191 podStartE2EDuration="2.226064001s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.75581ms" isStatefulPod=false podCreationTimestamp="2026-07-08 07:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:00:54.22482053 +0000 UTC m=+366.938469366" watchObservedRunningTime="2026-07-08 07:00:54.226064001 +0000 UTC m=+366.939712837" Jul 08 07:00:54.368870 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.368835 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c57b88868-d946s"] Jul 08 07:00:54.371433 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.371417 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.374640 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.374618 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 07:00:54.374640 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.374629 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 07:00:54.374814 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.374673 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rhsnw\"" Jul 08 07:00:54.374814 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.374620 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 07:00:54.375218 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.375199 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 07:00:54.375624 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.375608 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 07:00:54.380552 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.380534 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 07:00:54.388572 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.388554 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c57b88868-d946s"] Jul 08 07:00:54.436999 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.436968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-secret-telemeter-client\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.436999 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t5kv\" (UniqueName: \"kubernetes.io/projected/9b147bf7-f520-49cc-8d95-fefa173341ef-kube-api-access-5t5kv\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.437209 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-telemeter-client-tls\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.437209 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437135 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-serving-certs-ca-bundle\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.437209 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-metrics-client-ca\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.437312 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-federate-client-tls\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.437312 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.437312 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.437276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.537852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-serving-certs-ca-bundle\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.537852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537813 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-metrics-client-ca\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538073 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-federate-client-tls\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538073 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537879 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538073 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538073 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537915 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-secret-telemeter-client\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538073 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t5kv\" (UniqueName: \"kubernetes.io/projected/9b147bf7-f520-49cc-8d95-fefa173341ef-kube-api-access-5t5kv\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538073 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.537953 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-telemeter-client-tls\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538647 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.538616 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-serving-certs-ca-bundle\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538832 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.538803 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.538961 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.538900 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b147bf7-f520-49cc-8d95-fefa173341ef-metrics-client-ca\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.540463 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.540425 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.540627 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.540608 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-telemeter-client-tls\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.540692 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.540625 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-secret-telemeter-client\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.540763 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.540743 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9b147bf7-f520-49cc-8d95-fefa173341ef-federate-client-tls\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.546512 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.546496 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t5kv\" (UniqueName: \"kubernetes.io/projected/9b147bf7-f520-49cc-8d95-fefa173341ef-kube-api-access-5t5kv\") pod \"telemeter-client-c57b88868-d946s\" (UID: \"9b147bf7-f520-49cc-8d95-fefa173341ef\") " pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.681111 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.681082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" Jul 08 07:00:54.820403 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.820321 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c57b88868-d946s"] Jul 08 07:00:54.823179 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:00:54.823138 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b147bf7_f520_49cc_8d95_fefa173341ef.slice/crio-96c99eed5d35d21ca286039cd62bdc974817423b10bdb01ff7430e7cec74e8a0 WatchSource:0}: Error finding container 96c99eed5d35d21ca286039cd62bdc974817423b10bdb01ff7430e7cec74e8a0: Status 404 returned error can't find the container with id 96c99eed5d35d21ca286039cd62bdc974817423b10bdb01ff7430e7cec74e8a0 Jul 08 07:00:54.825605 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:54.825584 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 07:00:55.209123 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:55.209086 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" event={"ID":"9b147bf7-f520-49cc-8d95-fefa173341ef","Type":"ContainerStarted","Data":"96c99eed5d35d21ca286039cd62bdc974817423b10bdb01ff7430e7cec74e8a0"} Jul 08 07:00:57.219231 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:57.219201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" event={"ID":"9b147bf7-f520-49cc-8d95-fefa173341ef","Type":"ContainerStarted","Data":"28b0ef1db7aa185fd596a9d584c07b2d2a9a79553078af5bfa422432fffe37fd"} Jul 08 07:00:57.219606 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:57.219284 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" event={"ID":"9b147bf7-f520-49cc-8d95-fefa173341ef","Type":"ContainerStarted","Data":"5df44df01c3163585ce5843b1e3b42c50d0800b3b08be3fe16d4001f15576db2"} Jul 08 07:00:57.219606 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:57.219304 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" event={"ID":"9b147bf7-f520-49cc-8d95-fefa173341ef","Type":"ContainerStarted","Data":"2997c8ced78daa61eb194b6e023ac0ce9c952609824bf0f87d62c9cdd6cb1586"} Jul 08 07:00:57.240562 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:57.240511 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c57b88868-d946s" podStartSLOduration=1.04084471 podStartE2EDuration="3.24049419s" totalImagesPullingTime="2.19964948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:00:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:00:57.239730799 +0000 UTC m=+369.953379635" watchObservedRunningTime="2026-07-08 07:00:57.24049419 +0000 UTC m=+369.954143025" Jul 08 07:00:58.108567 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.108532 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75cd4d457c-s6hbp"] Jul 08 07:00:58.111025 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.111001 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.122323 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.122297 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75cd4d457c-s6hbp"] Jul 08 07:00:58.171779 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.171749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-config\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.171779 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.171784 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd7bp\" (UniqueName: \"kubernetes.io/projected/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-kube-api-access-fd7bp\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.171943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.171806 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-trusted-ca-bundle\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.171943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.171913 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-serving-cert\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.172025 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.171977 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-oauth-config\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.172059 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.172031 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-oauth-serving-cert\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.172059 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.172050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-service-ca\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273328 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-serving-cert\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273772 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273365 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-oauth-config\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273772 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273409 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-oauth-serving-cert\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273772 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-service-ca\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273772 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-config\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273772 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd7bp\" (UniqueName: \"kubernetes.io/projected/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-kube-api-access-fd7bp\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.273772 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.273540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-trusted-ca-bundle\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.274347 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.274319 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-oauth-serving-cert\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.274460 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.274424 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-service-ca\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.274589 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.274567 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-config\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.274962 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.274945 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-trusted-ca-bundle\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.275922 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.275903 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-serving-cert\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.276017 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.275948 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-oauth-config\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.282883 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.282857 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd7bp\" (UniqueName: \"kubernetes.io/projected/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-kube-api-access-fd7bp\") pod \"console-75cd4d457c-s6hbp\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.421329 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.421240 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:00:58.558529 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:58.558503 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75cd4d457c-s6hbp"] Jul 08 07:00:58.560794 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:00:58.560765 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2dec28_06e3_4508_a9b1_c3279ef68e6a.slice/crio-6ba8abd2483e256e7625529d47f2abab9891ba35571de5be541fa5b7b1f9a9a2 WatchSource:0}: Error finding container 6ba8abd2483e256e7625529d47f2abab9891ba35571de5be541fa5b7b1f9a9a2: Status 404 returned error can't find the container with id 6ba8abd2483e256e7625529d47f2abab9891ba35571de5be541fa5b7b1f9a9a2 Jul 08 07:00:59.228740 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:59.228710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cd4d457c-s6hbp" event={"ID":"1d2dec28-06e3-4508-a9b1-c3279ef68e6a","Type":"ContainerStarted","Data":"27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe"} Jul 08 07:00:59.228919 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:59.228754 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cd4d457c-s6hbp" event={"ID":"1d2dec28-06e3-4508-a9b1-c3279ef68e6a","Type":"ContainerStarted","Data":"6ba8abd2483e256e7625529d47f2abab9891ba35571de5be541fa5b7b1f9a9a2"} Jul 08 07:00:59.246850 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:00:59.246799 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75cd4d457c-s6hbp" podStartSLOduration=1.24678358 podStartE2EDuration="1.24678358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:00:59.245356701 +0000 UTC m=+371.959005537" watchObservedRunningTime="2026-07-08 07:00:59.24678358 +0000 UTC m=+371.960432416" Jul 08 07:01:01.606497 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:01.606460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 07:01:01.608801 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:01.608777 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350c83c0-715e-4925-8f3e-27ff51dee9b1-metrics-certs\") pod \"network-metrics-daemon-kmdj4\" (UID: \"350c83c0-715e-4925-8f3e-27ff51dee9b1\") " pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 07:01:01.822843 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:01.822811 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6plfv\"" Jul 08 07:01:01.831096 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:01.831075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmdj4" Jul 08 07:01:01.958071 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:01.958048 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kmdj4"] Jul 08 07:01:01.960808 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:01:01.960787 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350c83c0_715e_4925_8f3e_27ff51dee9b1.slice/crio-e66e42ac6b9695a950ebe02c5edd7ae6f79fe27f0448f4e47f1da111f89bc654 WatchSource:0}: Error finding container e66e42ac6b9695a950ebe02c5edd7ae6f79fe27f0448f4e47f1da111f89bc654: Status 404 returned error can't find the container with id e66e42ac6b9695a950ebe02c5edd7ae6f79fe27f0448f4e47f1da111f89bc654 Jul 08 07:01:02.242739 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:02.242705 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmdj4" event={"ID":"350c83c0-715e-4925-8f3e-27ff51dee9b1","Type":"ContainerStarted","Data":"e66e42ac6b9695a950ebe02c5edd7ae6f79fe27f0448f4e47f1da111f89bc654"} Jul 08 07:01:03.252801 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:03.251814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmdj4" event={"ID":"350c83c0-715e-4925-8f3e-27ff51dee9b1","Type":"ContainerStarted","Data":"3c2664617f030e28840c546cc5fbc6c0803e9dc788fd7328e771321e72a802c5"} Jul 08 07:01:04.257339 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:04.257299 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmdj4" event={"ID":"350c83c0-715e-4925-8f3e-27ff51dee9b1","Type":"ContainerStarted","Data":"7e319283ac138efc33b7d541564cf7061f6076799a27d1b515b69de4d322ba09"} Jul 08 07:01:04.274555 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:04.274507 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kmdj4" podStartSLOduration=375.121392263 podStartE2EDuration="6m16.274491973s" totalImagesPullingTime="1.15309971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:01:04.272753737 +0000 UTC m=+376.986402574" watchObservedRunningTime="2026-07-08 07:01:04.274491973 +0000 UTC m=+376.988140808" Jul 08 07:01:08.421788 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:08.421747 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:01:08.422189 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:08.421845 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:01:08.426362 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:08.426338 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:01:09.281422 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:09.281394 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:01:09.330454 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:09.330417 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cf656bd-8fxns"] Jul 08 07:01:34.354884 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.354831 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74cf656bd-8fxns" podUID="bcf3a159-50c9-46c3-8536-9499d3dd6e07" containerName="console" containerID="cri-o://5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c" gracePeriod=15 Jul 08 07:01:34.607267 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.607211 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 07:01:34.705039 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705004 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-config\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705256 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705065 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkbn5\" (UniqueName: \"kubernetes.io/projected/bcf3a159-50c9-46c3-8536-9499d3dd6e07-kube-api-access-bkbn5\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705256 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705089 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-serving-cert\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705256 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705107 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-trusted-ca-bundle\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705256 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705149 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-oauth-serving-cert\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705256 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705221 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-service-ca\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705525 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705347 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-oauth-config\") pod \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\" (UID: \"bcf3a159-50c9-46c3-8536-9499d3dd6e07\") " Jul 08 07:01:34.705608 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705583 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-config" (OuterVolumeSpecName: "console-config") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:01:34.705666 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705611 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:01:34.705666 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705655 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-service-ca" (OuterVolumeSpecName: "service-ca") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:01:34.705758 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705703 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-oauth-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:34.705758 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705722 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:34.705859 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.705799 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:01:34.707403 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.707374 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:01:34.707506 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.707381 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf3a159-50c9-46c3-8536-9499d3dd6e07-kube-api-access-bkbn5" (OuterVolumeSpecName: "kube-api-access-bkbn5") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "kube-api-access-bkbn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:01:34.707506 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.707441 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bcf3a159-50c9-46c3-8536-9499d3dd6e07" (UID: "bcf3a159-50c9-46c3-8536-9499d3dd6e07"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:01:34.806808 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.806772 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkbn5\" (UniqueName: \"kubernetes.io/projected/bcf3a159-50c9-46c3-8536-9499d3dd6e07-kube-api-access-bkbn5\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:34.806808 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.806803 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:34.806808 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.806813 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-trusted-ca-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:34.807028 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.806822 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcf3a159-50c9-46c3-8536-9499d3dd6e07-service-ca\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:34.807028 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:34.806830 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcf3a159-50c9-46c3-8536-9499d3dd6e07-console-oauth-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:01:35.383606 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.383568 2568 generic.go:356] "Generic (PLEG): container finished" podID="bcf3a159-50c9-46c3-8536-9499d3dd6e07" containerID="5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c" exitCode=0 Jul 08 07:01:35.384001 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.383615 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf656bd-8fxns" Jul 08 07:01:35.384001 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.383651 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf656bd-8fxns" event={"ID":"bcf3a159-50c9-46c3-8536-9499d3dd6e07","Type":"ContainerDied","Data":"5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c"} Jul 08 07:01:35.384001 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.383711 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf656bd-8fxns" event={"ID":"bcf3a159-50c9-46c3-8536-9499d3dd6e07","Type":"ContainerDied","Data":"9f0921f14f3a06bc0f5a04759bc526be3200805b3f49133796664596ea99eea0"} Jul 08 07:01:35.384001 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.383734 2568 scope.go:117] "RemoveContainer" containerID="5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c" Jul 08 07:01:35.392440 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.392424 2568 scope.go:117] "RemoveContainer" containerID="5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c" Jul 08 07:01:35.392673 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:01:35.392656 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c\": container with ID starting with 5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c not found: ID does not exist" containerID="5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c" Jul 08 07:01:35.392723 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.392684 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c"} err="failed to get container status \"5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c\": rpc error: code = NotFound desc = could not find container \"5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c\": container with ID starting with 5ee925571c97bb920be0351ccca68e69ed0a0b06d45cbc4f3b1048f40f9d149c not found: ID does not exist" Jul 08 07:01:35.403203 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.403180 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cf656bd-8fxns"] Jul 08 07:01:35.407709 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.407685 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74cf656bd-8fxns"] Jul 08 07:01:35.824911 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:01:35.824871 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf3a159-50c9-46c3-8536-9499d3dd6e07" path="/var/lib/kubelet/pods/bcf3a159-50c9-46c3-8536-9499d3dd6e07/volumes" Jul 08 07:02:07.624120 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.624088 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57878cb84b-n4wwq"] Jul 08 07:02:07.624592 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.624426 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf3a159-50c9-46c3-8536-9499d3dd6e07" containerName="console" Jul 08 07:02:07.624592 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.624439 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf3a159-50c9-46c3-8536-9499d3dd6e07" containerName="console" Jul 08 07:02:07.624592 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.624486 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf3a159-50c9-46c3-8536-9499d3dd6e07" containerName="console" Jul 08 07:02:07.629170 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.629141 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.645520 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.645495 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57878cb84b-n4wwq"] Jul 08 07:02:07.794132 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794094 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-serving-cert\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.794132 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-service-ca\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.794408 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794210 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-oauth-config\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.794408 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-oauth-serving-cert\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.794408 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49d7x\" (UniqueName: \"kubernetes.io/projected/5832bafa-d232-4e05-b25b-64f8495db70b-kube-api-access-49d7x\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.794531 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-console-config\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.794531 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.794448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-trusted-ca-bundle\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895319 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-oauth-serving-cert\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895319 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895274 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49d7x\" (UniqueName: \"kubernetes.io/projected/5832bafa-d232-4e05-b25b-64f8495db70b-kube-api-access-49d7x\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895319 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895301 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-console-config\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895622 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895438 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-trusted-ca-bundle\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895622 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-serving-cert\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895622 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-service-ca\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.895622 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.895580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-oauth-config\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.896105 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.896077 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-console-config\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.896228 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.896077 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-oauth-serving-cert\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.896228 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.896122 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-service-ca\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.896355 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.896339 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-trusted-ca-bundle\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.898059 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.898031 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-oauth-config\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.898251 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.898122 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-serving-cert\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.903499 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.903478 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49d7x\" (UniqueName: \"kubernetes.io/projected/5832bafa-d232-4e05-b25b-64f8495db70b-kube-api-access-49d7x\") pod \"console-57878cb84b-n4wwq\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:07.938218 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:07.938191 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:08.066108 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:08.066084 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57878cb84b-n4wwq"] Jul 08 07:02:08.068508 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:02:08.068479 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5832bafa_d232_4e05_b25b_64f8495db70b.slice/crio-0e6e281c6155cd5867c089f2981c5e01cec7d3c42378576d4e85a43932122ae8 WatchSource:0}: Error finding container 0e6e281c6155cd5867c089f2981c5e01cec7d3c42378576d4e85a43932122ae8: Status 404 returned error can't find the container with id 0e6e281c6155cd5867c089f2981c5e01cec7d3c42378576d4e85a43932122ae8 Jul 08 07:02:08.515043 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:08.515007 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57878cb84b-n4wwq" event={"ID":"5832bafa-d232-4e05-b25b-64f8495db70b","Type":"ContainerStarted","Data":"160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53"} Jul 08 07:02:08.515234 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:08.515052 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57878cb84b-n4wwq" event={"ID":"5832bafa-d232-4e05-b25b-64f8495db70b","Type":"ContainerStarted","Data":"0e6e281c6155cd5867c089f2981c5e01cec7d3c42378576d4e85a43932122ae8"} Jul 08 07:02:08.534049 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:08.534004 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57878cb84b-n4wwq" podStartSLOduration=1.533985537 podStartE2EDuration="1.533985537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:02:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:02:08.532720995 +0000 UTC m=+441.246369833" watchObservedRunningTime="2026-07-08 07:02:08.533985537 +0000 UTC m=+441.247634374" Jul 08 07:02:17.938612 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:17.938521 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:17.938612 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:17.938586 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:17.943607 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:17.943585 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:18.559113 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:18.559085 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:02:18.626667 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:18.626632 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75cd4d457c-s6hbp"] Jul 08 07:02:43.647352 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:43.647278 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-75cd4d457c-s6hbp" podUID="1d2dec28-06e3-4508-a9b1-c3279ef68e6a" containerName="console" containerID="cri-o://27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe" gracePeriod=15 Jul 08 07:02:43.888634 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:43.888612 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:02:44.011823 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.011777 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd7bp\" (UniqueName: \"kubernetes.io/projected/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-kube-api-access-fd7bp\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012027 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.011874 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-oauth-config\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012027 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.011906 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-oauth-serving-cert\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012027 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.011934 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-trusted-ca-bundle\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012027 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.011964 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-config\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012277 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012189 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-service-ca\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012339 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012284 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-serving-cert\") pod \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\" (UID: \"1d2dec28-06e3-4508-a9b1-c3279ef68e6a\") " Jul 08 07:02:44.012496 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012367 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:02:44.012496 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012485 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:02:44.012654 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012512 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-service-ca" (OuterVolumeSpecName: "service-ca") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:02:44.012654 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012525 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-config" (OuterVolumeSpecName: "console-config") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:02:44.012654 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012610 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-oauth-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.012654 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012627 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-trusted-ca-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.012654 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012638 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.012654 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.012646 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-service-ca\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.014056 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.014026 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:02:44.014056 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.014034 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-kube-api-access-fd7bp" (OuterVolumeSpecName: "kube-api-access-fd7bp") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "kube-api-access-fd7bp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:02:44.014279 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.014261 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1d2dec28-06e3-4508-a9b1-c3279ef68e6a" (UID: "1d2dec28-06e3-4508-a9b1-c3279ef68e6a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:02:44.113972 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.113922 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd7bp\" (UniqueName: \"kubernetes.io/projected/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-kube-api-access-fd7bp\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.113972 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.113967 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-oauth-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.113972 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.113976 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d2dec28-06e3-4508-a9b1-c3279ef68e6a-console-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:02:44.669198 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.669140 2568 generic.go:356] "Generic (PLEG): container finished" podID="1d2dec28-06e3-4508-a9b1-c3279ef68e6a" containerID="27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe" exitCode=0 Jul 08 07:02:44.669678 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.669209 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cd4d457c-s6hbp" Jul 08 07:02:44.669678 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.669228 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cd4d457c-s6hbp" event={"ID":"1d2dec28-06e3-4508-a9b1-c3279ef68e6a","Type":"ContainerDied","Data":"27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe"} Jul 08 07:02:44.669678 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.669281 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cd4d457c-s6hbp" event={"ID":"1d2dec28-06e3-4508-a9b1-c3279ef68e6a","Type":"ContainerDied","Data":"6ba8abd2483e256e7625529d47f2abab9891ba35571de5be541fa5b7b1f9a9a2"} Jul 08 07:02:44.669678 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.669312 2568 scope.go:117] "RemoveContainer" containerID="27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe" Jul 08 07:02:44.679648 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.679630 2568 scope.go:117] "RemoveContainer" containerID="27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe" Jul 08 07:02:44.679911 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:02:44.679892 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe\": container with ID starting with 27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe not found: ID does not exist" containerID="27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe" Jul 08 07:02:44.679980 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.679917 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe"} err="failed to get container status \"27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe\": rpc error: code = NotFound desc = could not find container \"27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe\": container with ID starting with 27225bf1853b444833f776f10b46192ca538506694a303662c70adca89fd9dbe not found: ID does not exist" Jul 08 07:02:44.692826 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.692803 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75cd4d457c-s6hbp"] Jul 08 07:02:44.698117 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:44.698097 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75cd4d457c-s6hbp"] Jul 08 07:02:45.824206 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:02:45.824177 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d2dec28-06e3-4508-a9b1-c3279ef68e6a" path="/var/lib/kubelet/pods/1d2dec28-06e3-4508-a9b1-c3279ef68e6a/volumes" Jul 08 07:03:06.122838 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.122801 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp"] Jul 08 07:03:06.123309 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.123286 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d2dec28-06e3-4508-a9b1-c3279ef68e6a" containerName="console" Jul 08 07:03:06.123309 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.123304 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d2dec28-06e3-4508-a9b1-c3279ef68e6a" containerName="console" Jul 08 07:03:06.123458 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.123443 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d2dec28-06e3-4508-a9b1-c3279ef68e6a" containerName="console" Jul 08 07:03:06.128029 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.128008 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.130929 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.130900 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 07:03:06.131064 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.130978 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8p8tg\"" Jul 08 07:03:06.131721 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.131703 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 07:03:06.135207 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.134943 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp"] Jul 08 07:03:06.178651 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.178622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/a31c55bf-4b27-43d1-afed-576c99a97777-kube-api-access-54cfj\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.178793 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.178684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.178793 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.178759 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.279528 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.279494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.279689 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.279537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.279689 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.279595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/a31c55bf-4b27-43d1-afed-576c99a97777-kube-api-access-54cfj\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.279938 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.279919 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.279980 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.279954 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.287597 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.287572 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/a31c55bf-4b27-43d1-afed-576c99a97777-kube-api-access-54cfj\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.437837 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.437753 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:06.563045 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.562990 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp"] Jul 08 07:03:06.565170 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:03:06.565135 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31c55bf_4b27_43d1_afed_576c99a97777.slice/crio-4a31eb56a053bca5907f3e0334d743fcbfb3bc13214302f457bdfa26e8249de0 WatchSource:0}: Error finding container 4a31eb56a053bca5907f3e0334d743fcbfb3bc13214302f457bdfa26e8249de0: Status 404 returned error can't find the container with id 4a31eb56a053bca5907f3e0334d743fcbfb3bc13214302f457bdfa26e8249de0 Jul 08 07:03:06.752271 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:06.752237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" event={"ID":"a31c55bf-4b27-43d1-afed-576c99a97777","Type":"ContainerStarted","Data":"4a31eb56a053bca5907f3e0334d743fcbfb3bc13214302f457bdfa26e8249de0"} Jul 08 07:03:15.792041 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:15.791950 2568 generic.go:356] "Generic (PLEG): container finished" podID="a31c55bf-4b27-43d1-afed-576c99a97777" containerID="84fca0fc9dcef9f0c09be00114f5ac78fcc0a89697c76da648098a643d2263cd" exitCode=0 Jul 08 07:03:15.792492 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:15.792037 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" event={"ID":"a31c55bf-4b27-43d1-afed-576c99a97777","Type":"ContainerDied","Data":"84fca0fc9dcef9f0c09be00114f5ac78fcc0a89697c76da648098a643d2263cd"} Jul 08 07:03:17.802321 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:17.802287 2568 generic.go:356] "Generic (PLEG): container finished" podID="a31c55bf-4b27-43d1-afed-576c99a97777" containerID="6228ab44e5ac7f6a2cab7e941793f21374265b3c7edd6663e062bb2e181e596b" exitCode=0 Jul 08 07:03:17.802692 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:17.802372 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" event={"ID":"a31c55bf-4b27-43d1-afed-576c99a97777","Type":"ContainerDied","Data":"6228ab44e5ac7f6a2cab7e941793f21374265b3c7edd6663e062bb2e181e596b"} Jul 08 07:03:27.845465 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:27.845384 2568 generic.go:356] "Generic (PLEG): container finished" podID="a31c55bf-4b27-43d1-afed-576c99a97777" containerID="153ccdd932d7d4b197c6fae9e557342cf8cbf72f8653453e7c0fbda7b8b27e6f" exitCode=0 Jul 08 07:03:27.845465 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:27.845453 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" event={"ID":"a31c55bf-4b27-43d1-afed-576c99a97777","Type":"ContainerDied","Data":"153ccdd932d7d4b197c6fae9e557342cf8cbf72f8653453e7c0fbda7b8b27e6f"} Jul 08 07:03:28.976556 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:28.976531 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:29.081250 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.081216 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-bundle\") pod \"a31c55bf-4b27-43d1-afed-576c99a97777\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " Jul 08 07:03:29.081435 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.081317 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/a31c55bf-4b27-43d1-afed-576c99a97777-kube-api-access-54cfj\") pod \"a31c55bf-4b27-43d1-afed-576c99a97777\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " Jul 08 07:03:29.081435 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.081364 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-util\") pod \"a31c55bf-4b27-43d1-afed-576c99a97777\" (UID: \"a31c55bf-4b27-43d1-afed-576c99a97777\") " Jul 08 07:03:29.081947 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.081907 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-bundle" (OuterVolumeSpecName: "bundle") pod "a31c55bf-4b27-43d1-afed-576c99a97777" (UID: "a31c55bf-4b27-43d1-afed-576c99a97777"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 07:03:29.083464 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.083444 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a31c55bf-4b27-43d1-afed-576c99a97777-kube-api-access-54cfj" (OuterVolumeSpecName: "kube-api-access-54cfj") pod "a31c55bf-4b27-43d1-afed-576c99a97777" (UID: "a31c55bf-4b27-43d1-afed-576c99a97777"). InnerVolumeSpecName "kube-api-access-54cfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:03:29.085635 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.085611 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-util" (OuterVolumeSpecName: "util") pod "a31c55bf-4b27-43d1-afed-576c99a97777" (UID: "a31c55bf-4b27-43d1-afed-576c99a97777"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 07:03:29.182910 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.182822 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/a31c55bf-4b27-43d1-afed-576c99a97777-kube-api-access-54cfj\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:03:29.182910 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.182852 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-util\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:03:29.182910 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.182865 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a31c55bf-4b27-43d1-afed-576c99a97777-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:03:29.855383 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.855355 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" event={"ID":"a31c55bf-4b27-43d1-afed-576c99a97777","Type":"ContainerDied","Data":"4a31eb56a053bca5907f3e0334d743fcbfb3bc13214302f457bdfa26e8249de0"} Jul 08 07:03:29.855383 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.855371 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589qpwgp" Jul 08 07:03:29.855568 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:29.855400 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a31eb56a053bca5907f3e0334d743fcbfb3bc13214302f457bdfa26e8249de0" Jul 08 07:03:33.485252 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485211 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc"] Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485533 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="pull" Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485544 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="pull" Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485552 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="util" Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485558 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="util" Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485565 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="extract" Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485570 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="extract" Jul 08 07:03:33.485705 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.485634 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a31c55bf-4b27-43d1-afed-576c99a97777" containerName="extract" Jul 08 07:03:33.516437 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.516404 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc"] Jul 08 07:03:33.516591 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.516522 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.518992 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.518968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 08 07:03:33.519116 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.519068 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 08 07:03:33.519116 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.519104 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-ljrw9\"" Jul 08 07:03:33.519277 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.519072 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 08 07:03:33.622885 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.622850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/3c1cb82a-5dad-48d3-8d42-391994dfe949-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc\" (UID: \"3c1cb82a-5dad-48d3-8d42-391994dfe949\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.622885 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.622890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mmh5\" (UniqueName: \"kubernetes.io/projected/3c1cb82a-5dad-48d3-8d42-391994dfe949-kube-api-access-2mmh5\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc\" (UID: \"3c1cb82a-5dad-48d3-8d42-391994dfe949\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.724241 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.724206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/3c1cb82a-5dad-48d3-8d42-391994dfe949-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc\" (UID: \"3c1cb82a-5dad-48d3-8d42-391994dfe949\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.724425 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.724248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mmh5\" (UniqueName: \"kubernetes.io/projected/3c1cb82a-5dad-48d3-8d42-391994dfe949-kube-api-access-2mmh5\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc\" (UID: \"3c1cb82a-5dad-48d3-8d42-391994dfe949\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.726594 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.726566 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/3c1cb82a-5dad-48d3-8d42-391994dfe949-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc\" (UID: \"3c1cb82a-5dad-48d3-8d42-391994dfe949\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.736411 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.736350 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mmh5\" (UniqueName: \"kubernetes.io/projected/3c1cb82a-5dad-48d3-8d42-391994dfe949-kube-api-access-2mmh5\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc\" (UID: \"3c1cb82a-5dad-48d3-8d42-391994dfe949\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.826885 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.826862 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:33.958294 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:33.958264 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc"] Jul 08 07:03:33.961170 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:03:33.961120 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1cb82a_5dad_48d3_8d42_391994dfe949.slice/crio-705fd8ec003a1f7aa17d573d377ccdd8da75c29ddc8cfa6b0505d5d68f8e2e6c WatchSource:0}: Error finding container 705fd8ec003a1f7aa17d573d377ccdd8da75c29ddc8cfa6b0505d5d68f8e2e6c: Status 404 returned error can't find the container with id 705fd8ec003a1f7aa17d573d377ccdd8da75c29ddc8cfa6b0505d5d68f8e2e6c Jul 08 07:03:34.879033 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:34.878989 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" event={"ID":"3c1cb82a-5dad-48d3-8d42-391994dfe949","Type":"ContainerStarted","Data":"705fd8ec003a1f7aa17d573d377ccdd8da75c29ddc8cfa6b0505d5d68f8e2e6c"} Jul 08 07:03:36.890933 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:36.890849 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" event={"ID":"3c1cb82a-5dad-48d3-8d42-391994dfe949","Type":"ContainerStarted","Data":"0382814cbf58c66b8e99bfbdcbb064c9c0f1f65a2a59c532b140071dc7c1068d"} Jul 08 07:03:36.890933 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:36.890922 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:03:36.911486 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:36.911428 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" podStartSLOduration=1.318290087 podStartE2EDuration="3.911411496s" totalImagesPullingTime="2.593121409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:03:36.909078356 +0000 UTC m=+529.622727203" watchObservedRunningTime="2026-07-08 07:03:36.911411496 +0000 UTC m=+529.625060331" Jul 08 07:03:37.327675 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.327642 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-5bmh4"] Jul 08 07:03:37.330306 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.330281 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.332692 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.332672 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 08 07:03:37.332797 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.332708 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 08 07:03:37.332797 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.332720 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-2rfrz\"" Jul 08 07:03:37.341769 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.341744 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-5bmh4"] Jul 08 07:03:37.460296 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.460257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3185a29a-050a-4f1a-87e4-ae28558296c5-cabundle0\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.460477 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.460314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fg6p\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-kube-api-access-2fg6p\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.460477 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.460404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-certificates\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.561271 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.561236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-certificates\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.561435 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.561280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3185a29a-050a-4f1a-87e4-ae28558296c5-cabundle0\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.561435 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.561338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fg6p\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-kube-api-access-2fg6p\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.561435 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:03:37.561397 2568 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 08 07:03:37.561435 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:03:37.561418 2568 secret.go:281] references non-existent secret key: ca.crt Jul 08 07:03:37.561435 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:03:37.561430 2568 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 08 07:03:37.561703 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:03:37.561447 2568 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-5bmh4: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 08 07:03:37.561703 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:03:37.561502 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-certificates podName:3185a29a-050a-4f1a-87e4-ae28558296c5 nodeName:}" failed. No retries permitted until 2026-07-08 07:03:38.061483922 +0000 UTC m=+530.775132735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-certificates") pod "keda-operator-68bff59c-5bmh4" (UID: "3185a29a-050a-4f1a-87e4-ae28558296c5") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 08 07:03:37.562043 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.562018 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/3185a29a-050a-4f1a-87e4-ae28558296c5-cabundle0\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.570436 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.570404 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fg6p\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-kube-api-access-2fg6p\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:37.873686 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.873654 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8"] Jul 08 07:03:37.875927 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.875912 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:37.878208 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.878186 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 08 07:03:37.891175 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.891131 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8"] Jul 08 07:03:37.964712 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.964673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/6a3627bf-d9d6-492d-b535-0c6fb84f7578-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:37.964874 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.964739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns68g\" (UniqueName: \"kubernetes.io/projected/6a3627bf-d9d6-492d-b535-0c6fb84f7578-kube-api-access-ns68g\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:37.964874 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:37.964800 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a3627bf-d9d6-492d-b535-0c6fb84f7578-certificates\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.066104 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.066065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-certificates\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:38.066104 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.066108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a3627bf-d9d6-492d-b535-0c6fb84f7578-certificates\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.066317 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.066222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/6a3627bf-d9d6-492d-b535-0c6fb84f7578-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.066317 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.066307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns68g\" (UniqueName: \"kubernetes.io/projected/6a3627bf-d9d6-492d-b535-0c6fb84f7578-kube-api-access-ns68g\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.066684 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.066656 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/6a3627bf-d9d6-492d-b535-0c6fb84f7578-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.068749 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.068723 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3185a29a-050a-4f1a-87e4-ae28558296c5-certificates\") pod \"keda-operator-68bff59c-5bmh4\" (UID: \"3185a29a-050a-4f1a-87e4-ae28558296c5\") " pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:38.068868 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.068852 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a3627bf-d9d6-492d-b535-0c6fb84f7578-certificates\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.073797 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.073775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns68g\" (UniqueName: \"kubernetes.io/projected/6a3627bf-d9d6-492d-b535-0c6fb84f7578-kube-api-access-ns68g\") pod \"keda-metrics-apiserver-559f75f947-g6jt8\" (UID: \"6a3627bf-d9d6-492d-b535-0c6fb84f7578\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.189252 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.189169 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:38.240455 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.240425 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:38.339591 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.339054 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8"] Jul 08 07:03:38.342706 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:03:38.342651 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3627bf_d9d6_492d_b535_0c6fb84f7578.slice/crio-cd55b4eb8eaff77eb08dbbc3e729817f2e60d1e0d6bc2bb5ebc39364fddce3ce WatchSource:0}: Error finding container cd55b4eb8eaff77eb08dbbc3e729817f2e60d1e0d6bc2bb5ebc39364fddce3ce: Status 404 returned error can't find the container with id cd55b4eb8eaff77eb08dbbc3e729817f2e60d1e0d6bc2bb5ebc39364fddce3ce Jul 08 07:03:38.391206 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.391179 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-5bmh4"] Jul 08 07:03:38.393674 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:03:38.393650 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3185a29a_050a_4f1a_87e4_ae28558296c5.slice/crio-c73be640010552c1bd348993880f9022660c4805e7659d3667262989e85533f3 WatchSource:0}: Error finding container c73be640010552c1bd348993880f9022660c4805e7659d3667262989e85533f3: Status 404 returned error can't find the container with id c73be640010552c1bd348993880f9022660c4805e7659d3667262989e85533f3 Jul 08 07:03:38.901098 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.901060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" event={"ID":"6a3627bf-d9d6-492d-b535-0c6fb84f7578","Type":"ContainerStarted","Data":"cd55b4eb8eaff77eb08dbbc3e729817f2e60d1e0d6bc2bb5ebc39364fddce3ce"} Jul 08 07:03:38.902270 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:38.902244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-5bmh4" event={"ID":"3185a29a-050a-4f1a-87e4-ae28558296c5","Type":"ContainerStarted","Data":"c73be640010552c1bd348993880f9022660c4805e7659d3667262989e85533f3"} Jul 08 07:03:42.926981 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:42.926946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" event={"ID":"6a3627bf-d9d6-492d-b535-0c6fb84f7578","Type":"ContainerStarted","Data":"a378413e44ed11b63fb17c6779e3fd38e97b48dda650b8a9362250682aaf551c"} Jul 08 07:03:42.927448 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:42.927101 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:42.928341 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:42.928321 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-5bmh4" event={"ID":"3185a29a-050a-4f1a-87e4-ae28558296c5","Type":"ContainerStarted","Data":"ad26a20ee7f86336063f1ae31797311b7ba914f53033ca0da69c3d642c3fcd2e"} Jul 08 07:03:42.928459 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:42.928445 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:03:42.945451 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:42.945408 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" podStartSLOduration=2.042976451 podStartE2EDuration="5.945393819s" totalImagesPullingTime="3.902417368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:03:42.943648555 +0000 UTC m=+535.657297415" watchObservedRunningTime="2026-07-08 07:03:42.945393819 +0000 UTC m=+535.659042655" Jul 08 07:03:42.971296 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:42.971253 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-5bmh4" podStartSLOduration=2.116246762 podStartE2EDuration="5.971241968s" totalImagesPullingTime="3.854995206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:03:42.969740336 +0000 UTC m=+535.683389185" watchObservedRunningTime="2026-07-08 07:03:42.971241968 +0000 UTC m=+535.684890804" Jul 08 07:03:53.937411 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:53.937380 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-g6jt8" Jul 08 07:03:57.898385 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:03:57.898356 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lq5hc" Jul 08 07:04:03.935191 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:03.935141 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-5bmh4" Jul 08 07:04:44.192010 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.191975 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-9487895bf-rv7wm"] Jul 08 07:04:44.194218 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.194199 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.196706 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.196687 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 08 07:04:44.197192 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.197166 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 08 07:04:44.197409 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.197390 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 08 07:04:44.197870 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.197856 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-vtrjz\"" Jul 08 07:04:44.205354 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.205333 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-rv7wm"] Jul 08 07:04:44.236249 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.236205 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blh7f\" (UniqueName: \"kubernetes.io/projected/acc3e8e5-6444-4eba-a444-09fcf009a96a-kube-api-access-blh7f\") pod \"kserve-controller-manager-9487895bf-rv7wm\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.236415 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.236276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acc3e8e5-6444-4eba-a444-09fcf009a96a-cert\") pod \"kserve-controller-manager-9487895bf-rv7wm\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.336791 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.336758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blh7f\" (UniqueName: \"kubernetes.io/projected/acc3e8e5-6444-4eba-a444-09fcf009a96a-kube-api-access-blh7f\") pod \"kserve-controller-manager-9487895bf-rv7wm\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.336964 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.336807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acc3e8e5-6444-4eba-a444-09fcf009a96a-cert\") pod \"kserve-controller-manager-9487895bf-rv7wm\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.339202 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.339180 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acc3e8e5-6444-4eba-a444-09fcf009a96a-cert\") pod \"kserve-controller-manager-9487895bf-rv7wm\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.345954 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.345930 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blh7f\" (UniqueName: \"kubernetes.io/projected/acc3e8e5-6444-4eba-a444-09fcf009a96a-kube-api-access-blh7f\") pod \"kserve-controller-manager-9487895bf-rv7wm\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.506746 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.506663 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:44.638852 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:44.638827 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-rv7wm"] Jul 08 07:04:44.641015 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:04:44.640987 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc3e8e5_6444_4eba_a444_09fcf009a96a.slice/crio-f1a8de44f2436cf0b37da8e6f116853ed427eb1266b2aee66e166ae5c54e5174 WatchSource:0}: Error finding container f1a8de44f2436cf0b37da8e6f116853ed427eb1266b2aee66e166ae5c54e5174: Status 404 returned error can't find the container with id f1a8de44f2436cf0b37da8e6f116853ed427eb1266b2aee66e166ae5c54e5174 Jul 08 07:04:45.198960 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:45.198928 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" event={"ID":"acc3e8e5-6444-4eba-a444-09fcf009a96a","Type":"ContainerStarted","Data":"f1a8de44f2436cf0b37da8e6f116853ed427eb1266b2aee66e166ae5c54e5174"} Jul 08 07:04:47.210460 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:47.210427 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" event={"ID":"acc3e8e5-6444-4eba-a444-09fcf009a96a","Type":"ContainerStarted","Data":"15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e"} Jul 08 07:04:47.210846 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:47.210526 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:04:47.228952 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:47.228895 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" podStartSLOduration=0.773049644 podStartE2EDuration="3.228879082s" totalImagesPullingTime="2.455829438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:04:47.225990655 +0000 UTC m=+599.939639491" watchObservedRunningTime="2026-07-08 07:04:47.228879082 +0000 UTC m=+599.942527917" Jul 08 07:04:49.225474 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:49.225436 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:04:49.231260 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:04:49.231240 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:05:18.219741 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:18.219661 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:05:20.024998 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.024965 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-rv7wm"] Jul 08 07:05:20.025422 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.025149 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" podUID="acc3e8e5-6444-4eba-a444-09fcf009a96a" containerName="manager" containerID="cri-o://15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e" gracePeriod=10 Jul 08 07:05:20.051000 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.050973 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-9487895bf-8mhtd"] Jul 08 07:05:20.054070 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.054054 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.062543 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.062516 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-8mhtd"] Jul 08 07:05:20.157360 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.157330 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lph7p\" (UniqueName: \"kubernetes.io/projected/3cd48442-3bbc-4dad-ae59-55d844418214-kube-api-access-lph7p\") pod \"kserve-controller-manager-9487895bf-8mhtd\" (UID: \"3cd48442-3bbc-4dad-ae59-55d844418214\") " pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.157509 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.157430 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cd48442-3bbc-4dad-ae59-55d844418214-cert\") pod \"kserve-controller-manager-9487895bf-8mhtd\" (UID: \"3cd48442-3bbc-4dad-ae59-55d844418214\") " pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.257931 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.257902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lph7p\" (UniqueName: \"kubernetes.io/projected/3cd48442-3bbc-4dad-ae59-55d844418214-kube-api-access-lph7p\") pod \"kserve-controller-manager-9487895bf-8mhtd\" (UID: \"3cd48442-3bbc-4dad-ae59-55d844418214\") " pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.258058 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.258003 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cd48442-3bbc-4dad-ae59-55d844418214-cert\") pod \"kserve-controller-manager-9487895bf-8mhtd\" (UID: \"3cd48442-3bbc-4dad-ae59-55d844418214\") " pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.260365 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.260341 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cd48442-3bbc-4dad-ae59-55d844418214-cert\") pod \"kserve-controller-manager-9487895bf-8mhtd\" (UID: \"3cd48442-3bbc-4dad-ae59-55d844418214\") " pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.261297 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.261280 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:05:20.266245 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.266225 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lph7p\" (UniqueName: \"kubernetes.io/projected/3cd48442-3bbc-4dad-ae59-55d844418214-kube-api-access-lph7p\") pod \"kserve-controller-manager-9487895bf-8mhtd\" (UID: \"3cd48442-3bbc-4dad-ae59-55d844418214\") " pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.358463 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.358378 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blh7f\" (UniqueName: \"kubernetes.io/projected/acc3e8e5-6444-4eba-a444-09fcf009a96a-kube-api-access-blh7f\") pod \"acc3e8e5-6444-4eba-a444-09fcf009a96a\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " Jul 08 07:05:20.358463 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.358414 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acc3e8e5-6444-4eba-a444-09fcf009a96a-cert\") pod \"acc3e8e5-6444-4eba-a444-09fcf009a96a\" (UID: \"acc3e8e5-6444-4eba-a444-09fcf009a96a\") " Jul 08 07:05:20.360599 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.360569 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acc3e8e5-6444-4eba-a444-09fcf009a96a-cert" (OuterVolumeSpecName: "cert") pod "acc3e8e5-6444-4eba-a444-09fcf009a96a" (UID: "acc3e8e5-6444-4eba-a444-09fcf009a96a"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:05:20.360796 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.360782 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acc3e8e5-6444-4eba-a444-09fcf009a96a-kube-api-access-blh7f" (OuterVolumeSpecName: "kube-api-access-blh7f") pod "acc3e8e5-6444-4eba-a444-09fcf009a96a" (UID: "acc3e8e5-6444-4eba-a444-09fcf009a96a"). InnerVolumeSpecName "kube-api-access-blh7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:05:20.364361 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.364335 2568 generic.go:356] "Generic (PLEG): container finished" podID="acc3e8e5-6444-4eba-a444-09fcf009a96a" containerID="15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e" exitCode=0 Jul 08 07:05:20.364486 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.364369 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" event={"ID":"acc3e8e5-6444-4eba-a444-09fcf009a96a","Type":"ContainerDied","Data":"15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e"} Jul 08 07:05:20.364486 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.364418 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" event={"ID":"acc3e8e5-6444-4eba-a444-09fcf009a96a","Type":"ContainerDied","Data":"f1a8de44f2436cf0b37da8e6f116853ed427eb1266b2aee66e166ae5c54e5174"} Jul 08 07:05:20.364486 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.364439 2568 scope.go:117] "RemoveContainer" containerID="15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e" Jul 08 07:05:20.364486 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.364378 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9487895bf-rv7wm" Jul 08 07:05:20.372733 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.372711 2568 scope.go:117] "RemoveContainer" containerID="15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e" Jul 08 07:05:20.373026 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:05:20.373005 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e\": container with ID starting with 15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e not found: ID does not exist" containerID="15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e" Jul 08 07:05:20.373119 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.373031 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e"} err="failed to get container status \"15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e\": rpc error: code = NotFound desc = could not find container \"15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e\": container with ID starting with 15cafd8201c49e8a721b9bc7640138f62151be34e0ec7e127a650aa2250db01e not found: ID does not exist" Jul 08 07:05:20.387538 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.387515 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-rv7wm"] Jul 08 07:05:20.390745 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.390722 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-rv7wm"] Jul 08 07:05:20.400283 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.400259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:20.459626 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.459597 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blh7f\" (UniqueName: \"kubernetes.io/projected/acc3e8e5-6444-4eba-a444-09fcf009a96a-kube-api-access-blh7f\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:05:20.459626 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.459625 2568 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acc3e8e5-6444-4eba-a444-09fcf009a96a-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:05:20.528725 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:20.528698 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-9487895bf-8mhtd"] Jul 08 07:05:20.530618 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:05:20.530592 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd48442_3bbc_4dad_ae59_55d844418214.slice/crio-e1ef93a2dcf6a771d3b4bc036ba933d0d8a5726c6b01a368edf468fab4dcaa63 WatchSource:0}: Error finding container e1ef93a2dcf6a771d3b4bc036ba933d0d8a5726c6b01a368edf468fab4dcaa63: Status 404 returned error can't find the container with id e1ef93a2dcf6a771d3b4bc036ba933d0d8a5726c6b01a368edf468fab4dcaa63 Jul 08 07:05:21.370686 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:21.370594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" event={"ID":"3cd48442-3bbc-4dad-ae59-55d844418214","Type":"ContainerStarted","Data":"524bcf36c7035c52b8bc6d8cfdfef774684f615b0a65ee221cce325af14d1cc2"} Jul 08 07:05:21.370686 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:21.370647 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" event={"ID":"3cd48442-3bbc-4dad-ae59-55d844418214","Type":"ContainerStarted","Data":"e1ef93a2dcf6a771d3b4bc036ba933d0d8a5726c6b01a368edf468fab4dcaa63"} Jul 08 07:05:21.370686 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:21.370687 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:21.387571 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:21.387523 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" podStartSLOduration=0.870236967 podStartE2EDuration="1.387503672s" totalImagesPullingTime="517.266705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:05:21.387314842 +0000 UTC m=+634.100963691" watchObservedRunningTime="2026-07-08 07:05:21.387503672 +0000 UTC m=+634.101152509" Jul 08 07:05:21.825423 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:21.825392 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acc3e8e5-6444-4eba-a444-09fcf009a96a" path="/var/lib/kubelet/pods/acc3e8e5-6444-4eba-a444-09fcf009a96a/volumes" Jul 08 07:05:52.380723 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:52.380690 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-9487895bf-8mhtd" Jul 08 07:05:53.349093 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.349060 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-wmc9x"] Jul 08 07:05:53.349422 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.349408 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc3e8e5-6444-4eba-a444-09fcf009a96a" containerName="manager" Jul 08 07:05:53.349467 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.349424 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc3e8e5-6444-4eba-a444-09fcf009a96a" containerName="manager" Jul 08 07:05:53.349504 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.349493 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="acc3e8e5-6444-4eba-a444-09fcf009a96a" containerName="manager" Jul 08 07:05:53.353704 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.353684 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:53.356284 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.356262 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 08 07:05:53.356431 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.356313 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-ktmgq\"" Jul 08 07:05:53.362121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.362101 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-wmc9x"] Jul 08 07:05:53.363841 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.363820 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-4tmnv"] Jul 08 07:05:53.367090 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.367069 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:53.369610 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.369591 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-tg265\"" Jul 08 07:05:53.369709 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.369639 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 08 07:05:53.381140 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.381120 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-4tmnv"] Jul 08 07:05:53.448000 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.447968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08ae5a8e-da27-4e54-8b18-701083620bec-tls-certs\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:53.448192 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.448020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz8rc\" (UniqueName: \"kubernetes.io/projected/08ae5a8e-da27-4e54-8b18-701083620bec-kube-api-access-lz8rc\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:53.448192 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.448059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69d95097-72ff-4733-91be-b661269d14b5-cert\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:53.448192 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.448105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdlbf\" (UniqueName: \"kubernetes.io/projected/69d95097-72ff-4733-91be-b661269d14b5-kube-api-access-jdlbf\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:53.548582 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.548550 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz8rc\" (UniqueName: \"kubernetes.io/projected/08ae5a8e-da27-4e54-8b18-701083620bec-kube-api-access-lz8rc\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:53.548755 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.548590 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69d95097-72ff-4733-91be-b661269d14b5-cert\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:53.548755 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.548626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdlbf\" (UniqueName: \"kubernetes.io/projected/69d95097-72ff-4733-91be-b661269d14b5-kube-api-access-jdlbf\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:53.548755 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.548678 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08ae5a8e-da27-4e54-8b18-701083620bec-tls-certs\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:53.548917 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:05:53.548777 2568 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 07:05:53.548917 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:05:53.548777 2568 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 08 07:05:53.548917 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:05:53.548845 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69d95097-72ff-4733-91be-b661269d14b5-cert podName:69d95097-72ff-4733-91be-b661269d14b5 nodeName:}" failed. No retries permitted until 2026-07-08 07:05:54.048824504 +0000 UTC m=+666.762473331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69d95097-72ff-4733-91be-b661269d14b5-cert") pod "odh-model-controller-84b54b7f97-4tmnv" (UID: "69d95097-72ff-4733-91be-b661269d14b5") : secret "odh-model-controller-webhook-cert" not found Jul 08 07:05:53.548917 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:05:53.548899 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ae5a8e-da27-4e54-8b18-701083620bec-tls-certs podName:08ae5a8e-da27-4e54-8b18-701083620bec nodeName:}" failed. No retries permitted until 2026-07-08 07:05:54.04888043 +0000 UTC m=+666.762529251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/08ae5a8e-da27-4e54-8b18-701083620bec-tls-certs") pod "model-serving-api-fd65d7d6-wmc9x" (UID: "08ae5a8e-da27-4e54-8b18-701083620bec") : secret "model-serving-api-tls" not found Jul 08 07:05:53.560031 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.560006 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdlbf\" (UniqueName: \"kubernetes.io/projected/69d95097-72ff-4733-91be-b661269d14b5-kube-api-access-jdlbf\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:53.560244 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:53.560225 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz8rc\" (UniqueName: \"kubernetes.io/projected/08ae5a8e-da27-4e54-8b18-701083620bec-kube-api-access-lz8rc\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:54.052993 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.052956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69d95097-72ff-4733-91be-b661269d14b5-cert\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:54.053243 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.053046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08ae5a8e-da27-4e54-8b18-701083620bec-tls-certs\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:54.055507 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.055484 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08ae5a8e-da27-4e54-8b18-701083620bec-tls-certs\") pod \"model-serving-api-fd65d7d6-wmc9x\" (UID: \"08ae5a8e-da27-4e54-8b18-701083620bec\") " pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:54.055636 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.055615 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69d95097-72ff-4733-91be-b661269d14b5-cert\") pod \"odh-model-controller-84b54b7f97-4tmnv\" (UID: \"69d95097-72ff-4733-91be-b661269d14b5\") " pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:54.265625 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.265590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:54.277788 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.277764 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:54.409711 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.409676 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-wmc9x"] Jul 08 07:05:54.411771 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:05:54.411746 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ae5a8e_da27_4e54_8b18_701083620bec.slice/crio-3074df7f9b1c9b592d35c35e71fd08e959719d06daab8bc5c5195db3b2b97ade WatchSource:0}: Error finding container 3074df7f9b1c9b592d35c35e71fd08e959719d06daab8bc5c5195db3b2b97ade: Status 404 returned error can't find the container with id 3074df7f9b1c9b592d35c35e71fd08e959719d06daab8bc5c5195db3b2b97ade Jul 08 07:05:54.433733 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.433708 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-4tmnv"] Jul 08 07:05:54.436434 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:05:54.436410 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69d95097_72ff_4733_91be_b661269d14b5.slice/crio-b92684c9f688e86d6d7f6c9f204c777108264d140047a010c31085a0df510c6a WatchSource:0}: Error finding container b92684c9f688e86d6d7f6c9f204c777108264d140047a010c31085a0df510c6a: Status 404 returned error can't find the container with id b92684c9f688e86d6d7f6c9f204c777108264d140047a010c31085a0df510c6a Jul 08 07:05:54.519174 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.519110 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-wmc9x" event={"ID":"08ae5a8e-da27-4e54-8b18-701083620bec","Type":"ContainerStarted","Data":"3074df7f9b1c9b592d35c35e71fd08e959719d06daab8bc5c5195db3b2b97ade"} Jul 08 07:05:54.520252 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:54.520224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" event={"ID":"69d95097-72ff-4733-91be-b661269d14b5","Type":"ContainerStarted","Data":"b92684c9f688e86d6d7f6c9f204c777108264d140047a010c31085a0df510c6a"} Jul 08 07:05:56.537985 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:56.537944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-wmc9x" event={"ID":"08ae5a8e-da27-4e54-8b18-701083620bec","Type":"ContainerStarted","Data":"d3e434f1eb3a2393bfa0a2b3de324d52967d4e19207a652ed17ee02646759109"} Jul 08 07:05:56.538471 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:56.538087 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:05:56.554095 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:56.554044 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-wmc9x" podStartSLOduration=1.947233694 podStartE2EDuration="3.554028738s" totalImagesPullingTime="1.606795044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:05:56.552826382 +0000 UTC m=+669.266475225" watchObservedRunningTime="2026-07-08 07:05:56.554028738 +0000 UTC m=+669.267677575" Jul 08 07:05:58.549235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:58.549198 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" event={"ID":"69d95097-72ff-4733-91be-b661269d14b5","Type":"ContainerStarted","Data":"6226db670612c51f1a02fd5b620b764a6d44fed1f4e3ebdeef9ba6fa41d42385"} Jul 08 07:05:58.549677 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:58.549341 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:05:58.566964 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:58.566912 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" podStartSLOduration=2.440013854 podStartE2EDuration="5.566895234s" totalImagesPullingTime="3.12688138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:05:58.565372462 +0000 UTC m=+671.279021333" watchObservedRunningTime="2026-07-08 07:05:58.566895234 +0000 UTC m=+671.280544071" Jul 08 07:05:59.086963 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.086931 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76c666696-gtjwn"] Jul 08 07:05:59.090913 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.090889 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.101067 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.101039 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76c666696-gtjwn"] Jul 08 07:05:59.202227 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-trusted-ca-bundle\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.202386 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202245 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-oauth-serving-cert\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.202386 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-oauth-config\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.202386 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5n55\" (UniqueName: \"kubernetes.io/projected/c4c824f6-b98d-45c7-8d20-aa758c9ce004-kube-api-access-j5n55\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.202386 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-config\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.202386 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-service-ca\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.202552 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.202396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-serving-cert\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303709 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5n55\" (UniqueName: \"kubernetes.io/projected/c4c824f6-b98d-45c7-8d20-aa758c9ce004-kube-api-access-j5n55\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303709 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-config\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-service-ca\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303759 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-serving-cert\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-trusted-ca-bundle\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303834 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-oauth-serving-cert\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.303943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.303888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-oauth-config\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.304573 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.304540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-config\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.304573 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.304559 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-service-ca\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.304739 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.304693 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-oauth-serving-cert\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.304739 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.304714 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c824f6-b98d-45c7-8d20-aa758c9ce004-trusted-ca-bundle\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.306238 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.306214 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-oauth-config\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.306517 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.306502 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4c824f6-b98d-45c7-8d20-aa758c9ce004-console-serving-cert\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.311871 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.311843 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5n55\" (UniqueName: \"kubernetes.io/projected/c4c824f6-b98d-45c7-8d20-aa758c9ce004-kube-api-access-j5n55\") pod \"console-76c666696-gtjwn\" (UID: \"c4c824f6-b98d-45c7-8d20-aa758c9ce004\") " pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.402020 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.401919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:05:59.541922 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.541819 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76c666696-gtjwn"] Jul 08 07:05:59.544412 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:05:59.544382 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c824f6_b98d_45c7_8d20_aa758c9ce004.slice/crio-9a106419403489a51430f4d218b1e6af073c467a35408ab9cb0bdd626b15b9a7 WatchSource:0}: Error finding container 9a106419403489a51430f4d218b1e6af073c467a35408ab9cb0bdd626b15b9a7: Status 404 returned error can't find the container with id 9a106419403489a51430f4d218b1e6af073c467a35408ab9cb0bdd626b15b9a7 Jul 08 07:05:59.555354 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:05:59.555318 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76c666696-gtjwn" event={"ID":"c4c824f6-b98d-45c7-8d20-aa758c9ce004","Type":"ContainerStarted","Data":"9a106419403489a51430f4d218b1e6af073c467a35408ab9cb0bdd626b15b9a7"} Jul 08 07:06:00.561762 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:00.561723 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76c666696-gtjwn" event={"ID":"c4c824f6-b98d-45c7-8d20-aa758c9ce004","Type":"ContainerStarted","Data":"8ba99e638dcb34bb9b3e59d2e66443b40200fcbc8bc9b3ee2f131c11d946062c"} Jul 08 07:06:00.580631 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:00.580578 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76c666696-gtjwn" podStartSLOduration=1.580561434 podStartE2EDuration="1.580561434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:06:00.579462702 +0000 UTC m=+673.293111540" watchObservedRunningTime="2026-07-08 07:06:00.580561434 +0000 UTC m=+673.294210319" Jul 08 07:06:07.547878 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:07.547840 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-wmc9x" Jul 08 07:06:09.402520 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:09.402488 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:06:09.403005 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:09.402587 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:06:09.407502 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:09.407480 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:06:09.558459 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:09.558432 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-4tmnv" Jul 08 07:06:09.613019 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:09.612991 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76c666696-gtjwn" Jul 08 07:06:09.657072 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:09.656992 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57878cb84b-n4wwq"] Jul 08 07:06:34.683943 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:34.683880 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57878cb84b-n4wwq" podUID="5832bafa-d232-4e05-b25b-64f8495db70b" containerName="console" containerID="cri-o://160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53" gracePeriod=15 Jul 08 07:06:34.930729 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:34.930704 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:06:35.011738 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011703 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-console-config\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.011902 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011751 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-service-ca\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.011902 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011786 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-oauth-config\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.011902 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011829 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-trusted-ca-bundle\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.011902 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011851 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-oauth-serving-cert\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.011902 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011893 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49d7x\" (UniqueName: \"kubernetes.io/projected/5832bafa-d232-4e05-b25b-64f8495db70b-kube-api-access-49d7x\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.012203 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.011933 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-serving-cert\") pod \"5832bafa-d232-4e05-b25b-64f8495db70b\" (UID: \"5832bafa-d232-4e05-b25b-64f8495db70b\") " Jul 08 07:06:35.012271 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.012232 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-console-config" (OuterVolumeSpecName: "console-config") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:06:35.012322 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.012268 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-service-ca" (OuterVolumeSpecName: "service-ca") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:06:35.012449 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.012425 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:06:35.012502 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.012479 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:06:35.014121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.014088 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:06:35.014121 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.014105 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5832bafa-d232-4e05-b25b-64f8495db70b-kube-api-access-49d7x" (OuterVolumeSpecName: "kube-api-access-49d7x") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "kube-api-access-49d7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 07:06:35.014356 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.014124 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5832bafa-d232-4e05-b25b-64f8495db70b" (UID: "5832bafa-d232-4e05-b25b-64f8495db70b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:06:35.113181 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113128 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-service-ca\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.113181 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113183 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-oauth-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.113390 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113196 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-trusted-ca-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.113390 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113205 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-oauth-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.113390 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113214 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49d7x\" (UniqueName: \"kubernetes.io/projected/5832bafa-d232-4e05-b25b-64f8495db70b-kube-api-access-49d7x\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.113390 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113225 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5832bafa-d232-4e05-b25b-64f8495db70b-console-serving-cert\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.113390 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.113233 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5832bafa-d232-4e05-b25b-64f8495db70b-console-config\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:06:35.729551 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.729511 2568 generic.go:356] "Generic (PLEG): container finished" podID="5832bafa-d232-4e05-b25b-64f8495db70b" containerID="160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53" exitCode=0 Jul 08 07:06:35.730016 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.729562 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57878cb84b-n4wwq" Jul 08 07:06:35.730016 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.729596 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57878cb84b-n4wwq" event={"ID":"5832bafa-d232-4e05-b25b-64f8495db70b","Type":"ContainerDied","Data":"160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53"} Jul 08 07:06:35.730016 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.729658 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57878cb84b-n4wwq" event={"ID":"5832bafa-d232-4e05-b25b-64f8495db70b","Type":"ContainerDied","Data":"0e6e281c6155cd5867c089f2981c5e01cec7d3c42378576d4e85a43932122ae8"} Jul 08 07:06:35.730016 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.729705 2568 scope.go:117] "RemoveContainer" containerID="160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53" Jul 08 07:06:35.739477 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.739458 2568 scope.go:117] "RemoveContainer" containerID="160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53" Jul 08 07:06:35.739735 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:06:35.739715 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53\": container with ID starting with 160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53 not found: ID does not exist" containerID="160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53" Jul 08 07:06:35.739782 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.739745 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53"} err="failed to get container status \"160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53\": rpc error: code = NotFound desc = could not find container \"160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53\": container with ID starting with 160783977f1317519afc812ea74aeb112d5f3a4499535de0e426123eaf6b3b53 not found: ID does not exist" Jul 08 07:06:35.750233 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.750207 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57878cb84b-n4wwq"] Jul 08 07:06:35.753767 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.753747 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57878cb84b-n4wwq"] Jul 08 07:06:35.824725 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:06:35.824695 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5832bafa-d232-4e05-b25b-64f8495db70b" path="/var/lib/kubelet/pods/5832bafa-d232-4e05-b25b-64f8495db70b/volumes" Jul 08 07:09:49.252616 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:09:49.252505 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:09:49.259360 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:09:49.259336 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:10:56.518056 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.518024 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x"] Jul 08 07:10:56.518643 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.518586 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5832bafa-d232-4e05-b25b-64f8495db70b" containerName="console" Jul 08 07:10:56.518643 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.518606 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5832bafa-d232-4e05-b25b-64f8495db70b" containerName="console" Jul 08 07:10:56.518758 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.518705 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5832bafa-d232-4e05-b25b-64f8495db70b" containerName="console" Jul 08 07:10:56.521643 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.521623 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.524301 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.524266 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-a4509-serving-cert\"" Jul 08 07:10:56.524301 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.524275 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 08 07:10:56.524474 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.524385 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-a4509-kube-rbac-proxy-sar-config\"" Jul 08 07:10:56.525228 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.525211 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-f4gs4\"" Jul 08 07:10:56.536801 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.536778 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x"] Jul 08 07:10:56.618286 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.618249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93cd997-3d55-41e1-844d-3511aaca9cfa-openshift-service-ca-bundle\") pod \"model-chainer-raw-a4509-69b99bcd9d-ttw4x\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.618470 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.618329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e93cd997-3d55-41e1-844d-3511aaca9cfa-proxy-tls\") pod \"model-chainer-raw-a4509-69b99bcd9d-ttw4x\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.718957 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.718918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93cd997-3d55-41e1-844d-3511aaca9cfa-openshift-service-ca-bundle\") pod \"model-chainer-raw-a4509-69b99bcd9d-ttw4x\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.719128 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.718999 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e93cd997-3d55-41e1-844d-3511aaca9cfa-proxy-tls\") pod \"model-chainer-raw-a4509-69b99bcd9d-ttw4x\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.719591 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.719555 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93cd997-3d55-41e1-844d-3511aaca9cfa-openshift-service-ca-bundle\") pod \"model-chainer-raw-a4509-69b99bcd9d-ttw4x\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.721401 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.721383 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e93cd997-3d55-41e1-844d-3511aaca9cfa-proxy-tls\") pod \"model-chainer-raw-a4509-69b99bcd9d-ttw4x\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.832727 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.832625 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:56.966429 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.966406 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x"] Jul 08 07:10:56.968920 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:10:56.968886 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93cd997_3d55_41e1_844d_3511aaca9cfa.slice/crio-7ae21caab139e69271bba90be75d59f277b004e653ebf2b3eff2234625d849c3 WatchSource:0}: Error finding container 7ae21caab139e69271bba90be75d59f277b004e653ebf2b3eff2234625d849c3: Status 404 returned error can't find the container with id 7ae21caab139e69271bba90be75d59f277b004e653ebf2b3eff2234625d849c3 Jul 08 07:10:56.971441 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:56.971422 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 07:10:57.924548 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:57.924513 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" event={"ID":"e93cd997-3d55-41e1-844d-3511aaca9cfa","Type":"ContainerStarted","Data":"7ae21caab139e69271bba90be75d59f277b004e653ebf2b3eff2234625d849c3"} Jul 08 07:10:59.935854 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:59.935814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" event={"ID":"e93cd997-3d55-41e1-844d-3511aaca9cfa","Type":"ContainerStarted","Data":"44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b"} Jul 08 07:10:59.936240 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:59.935944 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:10:59.951792 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:10:59.951750 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podStartSLOduration=1.6502494109999999 podStartE2EDuration="3.951737398s" totalImagesPullingTime="2.301487987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:10:59.951450089 +0000 UTC m=+972.665098927" watchObservedRunningTime="2026-07-08 07:10:59.951737398 +0000 UTC m=+972.665386232" Jul 08 07:11:05.946373 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:05.946338 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:11:06.547262 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:06.547225 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x"] Jul 08 07:11:06.547449 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:06.547413 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" containerID="cri-o://44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b" gracePeriod=30 Jul 08 07:11:10.944750 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:10.944666 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:11:15.944752 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:15.944705 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:11:20.944233 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:20.944190 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:11:20.944635 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:20.944287 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:11:25.945560 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:25.945517 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:11:30.944530 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:30.944490 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:11:35.944384 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:35.944345 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:11:36.698666 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.698640 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:11:36.864577 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.864489 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93cd997-3d55-41e1-844d-3511aaca9cfa-openshift-service-ca-bundle\") pod \"e93cd997-3d55-41e1-844d-3511aaca9cfa\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " Jul 08 07:11:36.864577 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.864534 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e93cd997-3d55-41e1-844d-3511aaca9cfa-proxy-tls\") pod \"e93cd997-3d55-41e1-844d-3511aaca9cfa\" (UID: \"e93cd997-3d55-41e1-844d-3511aaca9cfa\") " Jul 08 07:11:36.864870 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.864846 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93cd997-3d55-41e1-844d-3511aaca9cfa-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e93cd997-3d55-41e1-844d-3511aaca9cfa" (UID: "e93cd997-3d55-41e1-844d-3511aaca9cfa"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:11:36.866628 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.866605 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e93cd997-3d55-41e1-844d-3511aaca9cfa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e93cd997-3d55-41e1-844d-3511aaca9cfa" (UID: "e93cd997-3d55-41e1-844d-3511aaca9cfa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:11:36.966025 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.965994 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e93cd997-3d55-41e1-844d-3511aaca9cfa-openshift-service-ca-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:11:36.966025 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:36.966021 2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e93cd997-3d55-41e1-844d-3511aaca9cfa-proxy-tls\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:11:37.103235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.103203 2568 generic.go:356] "Generic (PLEG): container finished" podID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerID="44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b" exitCode=0 Jul 08 07:11:37.103235 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.103248 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" Jul 08 07:11:37.103441 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.103285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" event={"ID":"e93cd997-3d55-41e1-844d-3511aaca9cfa","Type":"ContainerDied","Data":"44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b"} Jul 08 07:11:37.103441 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.103336 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x" event={"ID":"e93cd997-3d55-41e1-844d-3511aaca9cfa","Type":"ContainerDied","Data":"7ae21caab139e69271bba90be75d59f277b004e653ebf2b3eff2234625d849c3"} Jul 08 07:11:37.103441 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.103359 2568 scope.go:117] "RemoveContainer" containerID="44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b" Jul 08 07:11:37.111986 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.111968 2568 scope.go:117] "RemoveContainer" containerID="44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b" Jul 08 07:11:37.112246 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:11:37.112229 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b\": container with ID starting with 44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b not found: ID does not exist" containerID="44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b" Jul 08 07:11:37.112370 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.112255 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b"} err="failed to get container status \"44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b\": rpc error: code = NotFound desc = could not find container \"44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b\": container with ID starting with 44b1883d8fc5232875b4c0f7fd2d74479410b89ea0113386050a8569d5537a9b not found: ID does not exist" Jul 08 07:11:37.123752 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.123695 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x"] Jul 08 07:11:37.126873 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.126853 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a4509-69b99bcd9d-ttw4x"] Jul 08 07:11:37.826100 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:11:37.826060 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" path="/var/lib/kubelet/pods/e93cd997-3d55-41e1-844d-3511aaca9cfa/volumes" Jul 08 07:12:46.794778 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.794742 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4"] Jul 08 07:12:46.795262 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.795135 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" Jul 08 07:12:46.795262 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.795146 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" Jul 08 07:12:46.795262 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.795225 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e93cd997-3d55-41e1-844d-3511aaca9cfa" containerName="model-chainer-raw-a4509" Jul 08 07:12:46.798117 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.798098 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:46.800425 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.800399 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-aea5a-kube-rbac-proxy-sar-config\"" Jul 08 07:12:46.800601 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.800524 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 08 07:12:46.800664 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.800655 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-aea5a-serving-cert\"" Jul 08 07:12:46.800752 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.800730 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-f4gs4\"" Jul 08 07:12:46.808680 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.808656 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4"] Jul 08 07:12:46.857749 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.857713 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398fbe6e-9745-42da-a375-8b87fb80c26d-proxy-tls\") pod \"model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:46.857893 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.857772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/398fbe6e-9745-42da-a375-8b87fb80c26d-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:46.958669 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.958628 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/398fbe6e-9745-42da-a375-8b87fb80c26d-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:46.958857 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.958757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398fbe6e-9745-42da-a375-8b87fb80c26d-proxy-tls\") pod \"model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:46.959303 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.959283 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/398fbe6e-9745-42da-a375-8b87fb80c26d-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:46.961187 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:46.961148 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398fbe6e-9745-42da-a375-8b87fb80c26d-proxy-tls\") pod \"model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:47.109517 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:47.109429 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:47.244374 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:47.244344 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4"] Jul 08 07:12:47.246959 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:12:47.246934 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod398fbe6e_9745_42da_a375_8b87fb80c26d.slice/crio-76bcf405c8151ced913c1b5ef5bb6e8b8506e05c88c6e739f68c55bbd9576a14 WatchSource:0}: Error finding container 76bcf405c8151ced913c1b5ef5bb6e8b8506e05c88c6e739f68c55bbd9576a14: Status 404 returned error can't find the container with id 76bcf405c8151ced913c1b5ef5bb6e8b8506e05c88c6e739f68c55bbd9576a14 Jul 08 07:12:47.420763 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:47.420670 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" event={"ID":"398fbe6e-9745-42da-a375-8b87fb80c26d","Type":"ContainerStarted","Data":"f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4"} Jul 08 07:12:47.420763 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:47.420723 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" event={"ID":"398fbe6e-9745-42da-a375-8b87fb80c26d","Type":"ContainerStarted","Data":"76bcf405c8151ced913c1b5ef5bb6e8b8506e05c88c6e739f68c55bbd9576a14"} Jul 08 07:12:47.420763 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:47.420745 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:47.437403 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:47.437356 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podStartSLOduration=1.437341081 podStartE2EDuration="1.437341081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:12:47.435612388 +0000 UTC m=+1080.149261224" watchObservedRunningTime="2026-07-08 07:12:47.437341081 +0000 UTC m=+1080.150989916" Jul 08 07:12:53.431220 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:53.431193 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:12:56.849627 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:56.849593 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4"] Jul 08 07:12:56.850087 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:56.849780 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" containerID="cri-o://f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4" gracePeriod=30 Jul 08 07:12:58.429248 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:12:58.429199 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:13:03.430471 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:03.430426 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:13:08.429685 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:08.429641 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:13:08.430247 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:08.429760 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:13:13.429533 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:13.429490 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:13:18.436623 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:18.431213 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:13:23.430303 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:23.430263 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 07:13:26.993314 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:26.993285 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:13:27.101295 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.101267 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/398fbe6e-9745-42da-a375-8b87fb80c26d-openshift-service-ca-bundle\") pod \"398fbe6e-9745-42da-a375-8b87fb80c26d\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " Jul 08 07:13:27.101477 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.101307 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398fbe6e-9745-42da-a375-8b87fb80c26d-proxy-tls\") pod \"398fbe6e-9745-42da-a375-8b87fb80c26d\" (UID: \"398fbe6e-9745-42da-a375-8b87fb80c26d\") " Jul 08 07:13:27.101640 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.101614 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/398fbe6e-9745-42da-a375-8b87fb80c26d-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "398fbe6e-9745-42da-a375-8b87fb80c26d" (UID: "398fbe6e-9745-42da-a375-8b87fb80c26d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 07:13:27.103273 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.103255 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/398fbe6e-9745-42da-a375-8b87fb80c26d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "398fbe6e-9745-42da-a375-8b87fb80c26d" (UID: "398fbe6e-9745-42da-a375-8b87fb80c26d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 07:13:27.202891 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.202857 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/398fbe6e-9745-42da-a375-8b87fb80c26d-openshift-service-ca-bundle\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:13:27.202891 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.202887 2568 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/398fbe6e-9745-42da-a375-8b87fb80c26d-proxy-tls\") on node \"ip-10-0-138-98.ec2.internal\" DevicePath \"\"" Jul 08 07:13:27.608295 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.608261 2568 generic.go:356] "Generic (PLEG): container finished" podID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerID="f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4" exitCode=0 Jul 08 07:13:27.608470 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.608309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" event={"ID":"398fbe6e-9745-42da-a375-8b87fb80c26d","Type":"ContainerDied","Data":"f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4"} Jul 08 07:13:27.608470 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.608347 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" event={"ID":"398fbe6e-9745-42da-a375-8b87fb80c26d","Type":"ContainerDied","Data":"76bcf405c8151ced913c1b5ef5bb6e8b8506e05c88c6e739f68c55bbd9576a14"} Jul 08 07:13:27.608470 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.608367 2568 scope.go:117] "RemoveContainer" containerID="f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4" Jul 08 07:13:27.608470 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.608311 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4" Jul 08 07:13:27.616789 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.616770 2568 scope.go:117] "RemoveContainer" containerID="f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4" Jul 08 07:13:27.617030 ip-10-0-138-98 kubenswrapper[2568]: E0708 07:13:27.617013 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4\": container with ID starting with f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4 not found: ID does not exist" containerID="f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4" Jul 08 07:13:27.617091 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.617037 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4"} err="failed to get container status \"f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4\": rpc error: code = NotFound desc = could not find container \"f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4\": container with ID starting with f1e84d2c6f3e94d9f3f6764918c0dd114c796562151b1109b18b4687d79d60d4 not found: ID does not exist" Jul 08 07:13:27.630917 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.630890 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4"] Jul 08 07:13:27.635320 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.635295 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-aea5a-66fdcfdbb4-j89r4"] Jul 08 07:13:27.825849 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:13:27.825808 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" path="/var/lib/kubelet/pods/398fbe6e-9745-42da-a375-8b87fb80c26d/volumes" Jul 08 07:14:49.280655 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:14:49.280535 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:14:49.288381 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:14:49.288355 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:19:49.307233 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:19:49.307093 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:19:49.315778 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:19:49.315752 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:23:17.068476 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:17.068402 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5mkr5_dc2d1ac6-4fd6-4189-b443-308666150cfc/global-pull-secret-syncer/0.log" Jul 08 07:23:17.214233 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:17.214199 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qcf6t_da0da73e-1565-4cd1-a906-37b8316b63e6/konnectivity-agent/0.log" Jul 08 07:23:17.282018 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:17.281985 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-98.ec2.internal_cafb6815aa15ff96ffbd1f08c40a3af8/haproxy/0.log" Jul 08 07:23:21.171590 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.171559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/alertmanager/0.log" Jul 08 07:23:21.197126 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.197102 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/config-reloader/0.log" Jul 08 07:23:21.229758 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.229726 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/kube-rbac-proxy-web/0.log" Jul 08 07:23:21.263951 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.263928 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/kube-rbac-proxy/0.log" Jul 08 07:23:21.288260 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.288235 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/kube-rbac-proxy-metric/0.log" Jul 08 07:23:21.309683 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.309658 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/prom-label-proxy/0.log" Jul 08 07:23:21.333519 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.333491 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0645e724-4f52-4ff9-ad2c-5c9232a231b8/init-config-reloader/0.log" Jul 08 07:23:21.488334 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.488307 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7bfccb7cc4-ntfxg_3254d724-4d1d-4223-a780-9817b26f871c/metrics-server/0.log" Jul 08 07:23:21.725221 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.725130 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swwlb_27623fbf-ed7a-4647-9200-68f9f3c4eed8/node-exporter/0.log" Jul 08 07:23:21.745983 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.745958 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swwlb_27623fbf-ed7a-4647-9200-68f9f3c4eed8/kube-rbac-proxy/0.log" Jul 08 07:23:21.764385 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:21.764362 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swwlb_27623fbf-ed7a-4647-9200-68f9f3c4eed8/init-textfile/0.log" Jul 08 07:23:22.153987 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.153905 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c57b88868-d946s_9b147bf7-f520-49cc-8d95-fefa173341ef/telemeter-client/0.log" Jul 08 07:23:22.182266 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.182229 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c57b88868-d946s_9b147bf7-f520-49cc-8d95-fefa173341ef/reload/0.log" Jul 08 07:23:22.203783 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.203757 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c57b88868-d946s_9b147bf7-f520-49cc-8d95-fefa173341ef/kube-rbac-proxy/0.log" Jul 08 07:23:22.233018 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.232989 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-554b9948fc-sdpv2_3ae8ec52-0b94-4c87-ba8c-919f55575290/thanos-query/0.log" Jul 08 07:23:22.253435 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.253410 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-554b9948fc-sdpv2_3ae8ec52-0b94-4c87-ba8c-919f55575290/kube-rbac-proxy-web/0.log" Jul 08 07:23:22.271628 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.271608 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-554b9948fc-sdpv2_3ae8ec52-0b94-4c87-ba8c-919f55575290/kube-rbac-proxy/0.log" Jul 08 07:23:22.294851 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.294828 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-554b9948fc-sdpv2_3ae8ec52-0b94-4c87-ba8c-919f55575290/prom-label-proxy/0.log" Jul 08 07:23:22.317445 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.317421 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-554b9948fc-sdpv2_3ae8ec52-0b94-4c87-ba8c-919f55575290/kube-rbac-proxy-rules/0.log" Jul 08 07:23:22.336203 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:22.336182 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-554b9948fc-sdpv2_3ae8ec52-0b94-4c87-ba8c-919f55575290/kube-rbac-proxy-metrics/0.log" Jul 08 07:23:24.286328 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:24.286289 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/2.log" Jul 08 07:23:24.291863 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:24.291834 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-7bpgm_eccceb79-1a81-4fda-8f22-976288fe4e6b/console-operator/3.log" Jul 08 07:23:24.780488 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:24.780460 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76c666696-gtjwn_c4c824f6-b98d-45c7-8d20-aa758c9ce004/console/0.log" Jul 08 07:23:24.809692 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:24.809657 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-zb79f_9766fd0b-a971-4e8d-a126-01cb547a7f34/download-server/0.log" Jul 08 07:23:25.300327 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.300293 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-8qs27_219164e5-2321-4c35-b812-e9a244954db8/volume-data-source-validator/0.log" Jul 08 07:23:25.498102 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.498071 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7"] Jul 08 07:23:25.498465 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.498451 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" Jul 08 07:23:25.498518 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.498467 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" Jul 08 07:23:25.498555 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.498538 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="398fbe6e-9745-42da-a375-8b87fb80c26d" containerName="model-chainer-raw-hpa-aea5a" Jul 08 07:23:25.501612 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.501590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.504066 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.504044 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdzkf\"/\"kube-root-ca.crt\"" Jul 08 07:23:25.504295 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.504276 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdzkf\"/\"openshift-service-ca.crt\"" Jul 08 07:23:25.504395 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.504280 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdzkf\"/\"default-dockercfg-97tdw\"" Jul 08 07:23:25.511399 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.511376 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7"] Jul 08 07:23:25.537806 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.537774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-lib-modules\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.537949 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.537815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-podres\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.537949 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.537841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6jkh\" (UniqueName: \"kubernetes.io/projected/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-kube-api-access-h6jkh\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.537949 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.537864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-sys\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.538076 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.537979 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-proc\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639058 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.638965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-proc\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639058 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-lib-modules\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639058 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-podres\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639376 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639074 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6jkh\" (UniqueName: \"kubernetes.io/projected/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-kube-api-access-h6jkh\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639376 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-sys\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639376 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639099 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-proc\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639376 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639182 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-sys\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639376 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639241 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-podres\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.639376 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.639249 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-lib-modules\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.647243 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.647212 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6jkh\" (UniqueName: \"kubernetes.io/projected/6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9-kube-api-access-h6jkh\") pod \"perf-node-gather-daemonset-9b6b7\" (UID: \"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9\") " pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.812542 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.812508 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:25.943225 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:25.943199 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7"] Jul 08 07:23:25.945615 ip-10-0-138-98 kubenswrapper[2568]: W0708 07:23:25.945591 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6d1ac4dd_0dd9_40c7_8c07_22cf8cd149d9.slice/crio-28776883b037f19622441ea770d1cf376246a981ba20fb124cd1c24ed3267d36 WatchSource:0}: Error finding container 28776883b037f19622441ea770d1cf376246a981ba20fb124cd1c24ed3267d36: Status 404 returned error can't find the container with id 28776883b037f19622441ea770d1cf376246a981ba20fb124cd1c24ed3267d36 Jul 08 07:23:26.274594 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.274541 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t5vjh_e26c3381-ae7d-4f03-a66f-163d012fbb99/dns/0.log" Jul 08 07:23:26.294061 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.294038 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t5vjh_e26c3381-ae7d-4f03-a66f-163d012fbb99/kube-rbac-proxy/0.log" Jul 08 07:23:26.324701 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.324664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" event={"ID":"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9","Type":"ContainerStarted","Data":"dd10f5f2993b3e93debf26b463b56e0f27628ebc075cf6addf4d3bd6c892de88"} Jul 08 07:23:26.325143 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.324713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" event={"ID":"6d1ac4dd-0dd9-40c7-8c07-22cf8cd149d9","Type":"ContainerStarted","Data":"28776883b037f19622441ea770d1cf376246a981ba20fb124cd1c24ed3267d36"} Jul 08 07:23:26.325143 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.324784 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:26.340254 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.340208 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" podStartSLOduration=1.340189617 podStartE2EDuration="1.340189617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 07:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:23:26.339257893 +0000 UTC m=+1719.052906731" watchObservedRunningTime="2026-07-08 07:23:26.340189617 +0000 UTC m=+1719.053838455" Jul 08 07:23:26.361869 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.361842 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqjwx_af17d5fd-f771-4b83-9885-73f28261f87c/dns-node-resolver/0.log" Jul 08 07:23:26.809685 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.809662 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56986c54d8-scx9h_baaebca7-e610-436f-b708-ebe0ed54fbb7/registry/0.log" Jul 08 07:23:26.873703 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:26.873673 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q57p9_671a9f0d-a582-476a-968d-44a4e93192f7/node-ca/0.log" Jul 08 07:23:27.639501 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:27.639471 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85bc4dbc77-fklwm_75038f86-0af0-4e4c-9a38-40c81032fa3d/router/0.log" Jul 08 07:23:28.003335 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:28.003308 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2v9h2_fde777f5-9d61-4513-aa36-68fc4d4cd507/serve-healthcheck-canary/0.log" Jul 08 07:23:28.475377 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:28.475347 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l6766_d74a50ee-05f3-4bdc-b128-575325472098/kube-rbac-proxy/0.log" Jul 08 07:23:28.493690 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:28.493666 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l6766_d74a50ee-05f3-4bdc-b128-575325472098/exporter/0.log" Jul 08 07:23:28.512405 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:28.512374 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l6766_d74a50ee-05f3-4bdc-b128-575325472098/extractor/0.log" Jul 08 07:23:30.520917 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:30.520877 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-9487895bf-8mhtd_3cd48442-3bbc-4dad-ae59-55d844418214/manager/0.log" Jul 08 07:23:30.564810 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:30.564786 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-wmc9x_08ae5a8e-da27-4e54-8b18-701083620bec/server/0.log" Jul 08 07:23:30.649579 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:30.649544 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-4tmnv_69d95097-72ff-4733-91be-b661269d14b5/manager/0.log" Jul 08 07:23:32.339032 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:32.339005 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdzkf/perf-node-gather-daemonset-9b6b7" Jul 08 07:23:35.781828 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.781717 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-47vfx_e95e3a6e-fc04-4ed0-a54f-10ec01027c19/kube-multus/0.log" Jul 08 07:23:35.831981 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.831949 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/kube-multus-additional-cni-plugins/0.log" Jul 08 07:23:35.851847 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.851822 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/egress-router-binary-copy/0.log" Jul 08 07:23:35.871897 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.871863 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/cni-plugins/0.log" Jul 08 07:23:35.892358 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.892336 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/bond-cni-plugin/0.log" Jul 08 07:23:35.911876 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.911846 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/routeoverride-cni/0.log" Jul 08 07:23:35.931027 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.931005 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/whereabouts-cni-bincopy/0.log" Jul 08 07:23:35.949801 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:35.949777 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69mr4_13995368-b3df-40c6-a914-591ec9da1ded/whereabouts-cni/0.log" Jul 08 07:23:36.307723 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:36.307690 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kmdj4_350c83c0-715e-4925-8f3e-27ff51dee9b1/network-metrics-daemon/0.log" Jul 08 07:23:36.328411 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:36.328384 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kmdj4_350c83c0-715e-4925-8f3e-27ff51dee9b1/kube-rbac-proxy/0.log" Jul 08 07:23:37.187519 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.187495 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/ovn-controller/0.log" Jul 08 07:23:37.206560 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.206537 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/ovn-acl-logging/0.log" Jul 08 07:23:37.230952 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.230930 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/kube-rbac-proxy-node/0.log" Jul 08 07:23:37.250055 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.250028 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 07:23:37.268821 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.268800 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/northd/0.log" Jul 08 07:23:37.288189 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.288147 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/nbdb/0.log" Jul 08 07:23:37.308775 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.308754 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/sbdb/0.log" Jul 08 07:23:37.347806 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:37.347762 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4cknp_4743de47-58d7-412d-b1de-2d99d6eedad8/ovnkube-controller/0.log" Jul 08 07:23:38.727864 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:38.727825 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-r7zbh_80dbef4c-3d50-4bf2-be0b-d110603e4fc1/network-check-target-container/0.log" Jul 08 07:23:39.546424 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:39.546392 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-5t9nd_51b3fa95-4da3-454c-8026-2d6e9469fae5/iptables-alerter/0.log" Jul 08 07:23:40.210829 ip-10-0-138-98 kubenswrapper[2568]: I0708 07:23:40.210795 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-wp9fw_7e825faf-4777-4030-ac38-ff6f4c19cefa/tuned/0.log"