Jul 01 16:58:30.680249 ip-10-0-140-168 systemd[1]: Starting Kubernetes Kubelet... Jul 01 16:58:31.164508 ip-10-0-140-168 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:58:31.164508 ip-10-0-140-168 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 16:58:31.164508 ip-10-0-140-168 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:58:31.164508 ip-10-0-140-168 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 16:58:31.164508 ip-10-0-140-168 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:58:31.165509 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.165424 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 16:58:31.170451 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170434 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:58:31.170451 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170452 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170456 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170459 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170462 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170465 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170468 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170470 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170473 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170475 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170478 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170480 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170483 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170485 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170493 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170496 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170499 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170501 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170503 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170506 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170509 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:58:31.170545 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170512 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170514 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170517 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170519 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170522 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170524 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170527 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170529 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170531 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170534 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170536 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170538 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170541 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170543 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170546 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170549 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170552 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170555 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170557 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170560 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:58:31.171041 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170562 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170565 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170567 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170570 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170573 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170575 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170577 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170580 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170582 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170585 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170588 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170590 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170594 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170597 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170600 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170603 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170605 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170608 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170610 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:58:31.171556 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170613 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170615 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170618 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170620 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170623 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170625 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170627 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170631 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170633 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170636 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170638 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170641 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170643 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170646 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170648 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170651 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170653 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170656 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170659 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170661 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:58:31.172025 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170664 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170666 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170669 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170671 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170673 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170676 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170680 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170683 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170685 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170689 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170692 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170695 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170698 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170700 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170702 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170705 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170707 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170710 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.170712 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:58:31.172525 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171392 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171399 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171403 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171406 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171409 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171412 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171414 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171417 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171420 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171423 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171426 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171429 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171431 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171434 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171436 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171439 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171441 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171443 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171446 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171448 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:58:31.173001 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171451 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171453 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171456 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171458 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171461 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171463 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171466 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171468 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171470 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171473 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171475 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171478 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171481 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171483 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171486 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171488 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171491 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171494 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171496 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171499 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:58:31.173486 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171501 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171504 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171506 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171508 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171511 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171514 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171516 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171519 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171522 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171524 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171527 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171529 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171532 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171534 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171537 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171539 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171541 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171544 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171548 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:58:31.173977 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171551 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171553 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171556 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171558 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171562 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171566 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171570 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171573 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171576 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171579 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171581 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171584 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171586 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171588 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171591 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171593 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171595 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171598 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171601 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:58:31.174447 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171603 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171605 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171608 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171611 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171613 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171615 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171618 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171620 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171622 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171625 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171628 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171630 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171632 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171635 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171637 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171640 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171642 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171645 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171647 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171650 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:58:31.174926 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.171652 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172033 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172038 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172041 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172043 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172046 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172048 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172051 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172053 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172057 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172059 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172062 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172065 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172067 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172070 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172072 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172075 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172079 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172082 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:58:31.175429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172085 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172088 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172091 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172094 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172097 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172100 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172103 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172105 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172108 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172110 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172129 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172133 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172136 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172139 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172141 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172144 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172146 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172149 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172152 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:58:31.175877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172154 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172156 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172159 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172161 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172164 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172166 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172169 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172172 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172174 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172177 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172179 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172181 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172184 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172186 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172188 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172191 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172194 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172196 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172199 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172201 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:58:31.176412 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172205 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172207 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172210 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172212 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172215 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172217 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172219 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172222 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172224 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172226 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172229 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172231 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172234 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172236 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172239 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172243 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172248 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172251 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172253 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:58:31.176885 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172257 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172259 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172262 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172265 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172267 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172271 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172273 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172275 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172278 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172281 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172283 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172287 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172290 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172293 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172296 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172298 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172301 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172304 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172306 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172308 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:58:31.177398 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172311 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172313 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172315 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172378 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172392 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172398 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172402 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172406 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172409 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172414 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172418 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172421 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172423 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172427 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172430 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172433 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172436 2580 flags.go:64] FLAG: --cgroup-root="" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172439 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172441 2580 flags.go:64] FLAG: --client-ca-file="" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172444 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172447 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172450 2580 flags.go:64] FLAG: --cluster-domain="" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172453 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172456 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 16:58:31.177877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172459 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172462 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172467 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172470 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172473 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172476 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172479 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172482 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172485 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172487 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172490 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172494 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172499 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172501 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172504 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172507 2580 flags.go:64] FLAG: --enable-server="true" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172510 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172516 2580 flags.go:64] FLAG: --event-burst="100" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172519 2580 flags.go:64] FLAG: --event-qps="50" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172521 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172524 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172527 2580 flags.go:64] FLAG: --eviction-hard="" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172532 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172535 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172538 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 16:58:31.178474 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172541 2580 flags.go:64] FLAG: --eviction-soft="" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172543 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172546 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172549 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172552 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172554 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172557 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172560 2580 flags.go:64] FLAG: --feature-gates="" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172563 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172566 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172568 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172571 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172574 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172577 2580 flags.go:64] FLAG: --help="false" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172579 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172582 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172585 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172588 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172591 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172596 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172599 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172601 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172604 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 16:58:31.179086 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172607 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172611 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172613 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172616 2580 flags.go:64] FLAG: --kube-reserved="" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172619 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172621 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172624 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172628 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172630 2580 flags.go:64] FLAG: --lock-file="" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172633 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172636 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172639 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172645 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172647 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172650 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172653 2580 flags.go:64] FLAG: --logging-format="text" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172655 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172658 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172661 2580 flags.go:64] FLAG: --manifest-url="" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172664 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172667 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172670 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172674 2580 flags.go:64] FLAG: --max-pods="110" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172677 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172679 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 16:58:31.179696 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172682 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172685 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172687 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172691 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172694 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172701 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172704 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172707 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172711 2580 flags.go:64] FLAG: --pod-cidr="" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172713 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172718 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172721 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172724 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172726 2580 flags.go:64] FLAG: --port="10250" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172729 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172738 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e9475e43c71f0b26" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172741 2580 flags.go:64] FLAG: --qos-reserved="" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172744 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172746 2580 flags.go:64] FLAG: --register-node="true" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172749 2580 flags.go:64] FLAG: --register-with-taints="" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172753 2580 flags.go:64] FLAG: --registry-burst="10" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172755 2580 flags.go:64] FLAG: --registry-qps="5" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172758 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172760 2580 flags.go:64] FLAG: --reserved-memory="" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172764 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 16:58:31.180313 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172766 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172769 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172772 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172774 2580 flags.go:64] FLAG: --runonce="false" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172777 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172780 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172783 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172785 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172788 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172791 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172795 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172798 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172800 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172803 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172805 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172808 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172814 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172817 2580 flags.go:64] FLAG: --system-cgroups="" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172819 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172824 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172827 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172829 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172833 2580 flags.go:64] FLAG: --tls-min-version="" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172841 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172844 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 16:58:31.180938 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172846 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172849 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172852 2580 flags.go:64] FLAG: --v="2" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172855 2580 flags.go:64] FLAG: --version="false" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172859 2580 flags.go:64] FLAG: --vmodule="" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172862 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.172865 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172991 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172996 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.172998 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173001 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173004 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173007 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173009 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173012 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173015 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173018 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173023 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173025 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173028 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173030 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:58:31.181553 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173032 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173035 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173039 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173041 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173044 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173046 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173049 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173051 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173054 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173057 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173059 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173062 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173064 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173067 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173070 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173072 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173075 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173077 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173079 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173082 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:58:31.182094 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173084 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173086 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173089 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173091 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173093 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173096 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173098 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173102 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173108 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173111 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173114 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173116 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173119 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173121 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173125 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173127 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173130 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173133 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173136 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:58:31.182565 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173138 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173140 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173143 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173146 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173149 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173151 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173153 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173156 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173159 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173163 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173165 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173168 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173171 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173174 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173176 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173179 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173182 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173184 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173187 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:58:31.183043 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173190 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173192 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173196 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173199 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173203 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173207 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173210 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173213 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173217 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173219 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173222 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173225 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173228 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173231 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173233 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173236 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173238 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173240 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173246 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:58:31.183499 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173248 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173250 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173253 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173255 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173258 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173260 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173262 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.173265 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.174096 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181030 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181043 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181056 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181062 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181115 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181120 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:58:31.183982 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181124 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181127 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181130 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181133 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181136 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181138 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181141 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181143 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181146 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181149 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181152 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181156 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181159 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181161 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181164 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181166 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181172 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181174 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181177 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:58:31.184357 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181180 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181182 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181185 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181187 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181189 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181192 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181194 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181197 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181200 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181203 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181206 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181208 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181211 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181213 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181215 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181218 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181221 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181223 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181225 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181228 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:58:31.184816 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181230 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181232 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181235 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181237 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181240 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181242 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181244 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181248 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181250 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181252 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181255 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181257 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181260 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181262 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181265 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181267 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181270 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181272 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181275 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181277 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:58:31.185311 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181280 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181283 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181286 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181291 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181294 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181297 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181301 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181304 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181307 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181310 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181312 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181315 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181318 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181320 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181323 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181325 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181328 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181330 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181332 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:58:31.185780 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181335 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181337 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181339 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181342 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181345 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181347 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181350 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181352 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181355 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181357 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181360 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181362 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181364 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181367 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181370 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181372 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181375 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181377 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:58:31.186371 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181380 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181384 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181494 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181499 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181502 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181504 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181507 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181511 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181514 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181517 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181520 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181522 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181525 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181527 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:58:31.186797 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181530 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181532 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181534 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181537 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181540 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181542 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181545 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181548 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181550 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181552 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181555 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181557 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181560 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181564 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181567 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181570 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181573 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181576 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181579 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:58:31.187149 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181581 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181584 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181586 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181589 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181591 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181593 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181596 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181598 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181600 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181603 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181605 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181608 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181610 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181613 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181615 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181617 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181620 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181622 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181624 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181627 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:58:31.187595 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181629 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181632 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181634 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181636 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181639 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181641 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181644 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181646 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181649 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181651 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181654 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181656 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181659 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181661 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181663 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181666 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181668 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181670 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181673 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181675 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:58:31.188159 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181677 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181680 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181684 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181687 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181689 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181691 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181694 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181696 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181698 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181701 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181704 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181706 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181709 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181711 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181713 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181716 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181718 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181721 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181723 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:58:31.188807 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181726 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181728 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181730 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181733 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181735 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181737 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181740 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181742 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:31.181744 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181749 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.181837 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.183809 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.185306 2580 server.go:1019] "Starting client certificate rotation" Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.185409 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 16:58:31.189365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.185445 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 16:58:31.189907 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.189671 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 16:58:31.191111 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.191093 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 16:58:31.200823 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.200806 2580 log.go:25] "Validated CRI v1 runtime API" Jul 01 16:58:31.202850 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.202836 2580 log.go:25] "Validated CRI v1 image API" Jul 01 16:58:31.206027 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.205978 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 16:58:31.209604 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.209586 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9eecf84a-3c95-490c-921e-3a189edbf0c6:/dev/nvme0n1p3 eb26b1c9-ae97-4396-ba18-aafed6051b6a:/dev/nvme0n1p4] Jul 01 16:58:31.209653 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.209604 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 16:58:31.215709 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.215601 2580 manager.go:216] Machine: {Timestamp:2026-07-01 16:58:31.21418749 +0000 UTC m=+0.383202397 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3140147 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d975dbe767c455ec74b9727eaacff SystemUUID:ec2d975d-be76-7c45-5ec7-4b9727eaacff BootID:6d202bc4-ff52-4cf3-80ee-ae92721a65b6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7e:03:2a:9a:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7e:03:2a:9a:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:48:fd:00:7c:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 16:58:31.215709 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.215703 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 16:58:31.215906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.215836 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 16:58:31.219291 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.219276 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 16:58:31.219411 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.219380 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 16:58:31.219536 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.219414 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 16:58:31.221013 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.221001 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 16:58:31.221052 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.221016 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 16:58:31.221052 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.221024 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 16:58:31.221906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.221896 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 16:58:31.222703 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.222693 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 16:58:31.225998 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.225987 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 01 16:58:31.226109 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226100 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 16:58:31.226146 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226128 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 01 16:58:31.226146 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226138 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 16:58:31.226217 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226153 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 16:58:31.226217 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226163 2580 kubelet.go:398] "Adding apiserver pod source" Jul 01 16:58:31.226217 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226177 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 16:58:31.226515 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226499 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 16:58:31.226559 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.226518 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 16:58:31.232034 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.232017 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 16:58:31.234303 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.234290 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 16:58:31.234358 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.234319 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 16:58:31.234988 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.234977 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 16:58:31.235020 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.234994 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 16:58:31.235020 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.234999 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 16:58:31.235020 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235004 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 16:58:31.235020 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235012 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 16:58:31.235020 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235017 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 16:58:31.235179 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235171 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 16:58:31.235179 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235179 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 16:58:31.235230 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235185 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 16:58:31.235230 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235191 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 16:58:31.235230 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235199 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 16:58:31.235230 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.235206 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 16:58:31.236214 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.236204 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 16:58:31.236214 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.236214 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 16:58:31.238046 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.238030 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 16:58:31.238120 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.238097 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 16:58:31.238228 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.238212 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 16:58:31.239344 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.239334 2580 server.go:1268] "Started kubelet" Jul 01 16:58:31.239588 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.239458 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 16:58:31.239588 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.239475 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 16:58:31.239588 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.239530 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 16:58:31.240179 ip-10-0-140-168 systemd[1]: Started Kubernetes Kubelet. Jul 01 16:58:31.242087 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.242074 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 01 16:58:31.243056 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.243037 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 16:58:31.244033 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.244016 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 16:58:31.247570 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.247553 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 16:58:31.247570 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.247560 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 16:58:31.247725 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.247703 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.248353 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.248266 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 16:58:31.248353 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.248280 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 16:58:31.248486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.248397 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 01 16:58:31.248486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.248405 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 01 16:58:31.248573 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.248551 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 16:58:31.249472 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249452 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rx4jb" Jul 01 16:58:31.249691 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249677 2580 factory.go:153] Registering CRI-O factory Jul 01 16:58:31.249800 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249782 2580 factory.go:223] Registration of the crio container factory successfully Jul 01 16:58:31.249896 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249839 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 16:58:31.249896 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249851 2580 factory.go:55] Registering systemd factory Jul 01 16:58:31.249896 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249858 2580 factory.go:223] Registration of the systemd container factory successfully Jul 01 16:58:31.249896 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249878 2580 factory.go:103] Registering Raw factory Jul 01 16:58:31.249896 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.249892 2580 manager.go:1196] Started watching for new ooms in manager Jul 01 16:58:31.250326 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.250300 2580 manager.go:349] Starting recovery of all containers Jul 01 16:58:31.253831 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.253801 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 16:58:31.253916 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.253880 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 16:58:31.254590 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.253251 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-168.ec2.internal.18be38f13c8af2ca default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-168.ec2.internal,UID:ip-10-0-140-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-168.ec2.internal,},FirstTimestamp:2026-07-01 16:58:31.239307978 +0000 UTC m=+0.408322882,LastTimestamp:2026-07-01 16:58:31.239307978 +0000 UTC m=+0.408322882,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-168.ec2.internal,}" Jul 01 16:58:31.257145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.257086 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 16:58:31.259686 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.259660 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rx4jb" Jul 01 16:58:31.263232 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.263215 2580 manager.go:354] Recovery completed Jul 01 16:58:31.267344 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.267329 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.276055 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.276024 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.276122 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.276061 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.276122 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.276075 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.277791 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.277776 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 16:58:31.277791 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.277788 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 16:58:31.277899 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.277802 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 01 16:58:31.278182 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.278125 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-168.ec2.internal.18be38f13ebb8d1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-168.ec2.internal,UID:ip-10-0-140-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-168.ec2.internal,},FirstTimestamp:2026-07-01 16:58:31.276047646 +0000 UTC m=+0.445062555,LastTimestamp:2026-07-01 16:58:31.276047646 +0000 UTC m=+0.445062555,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-168.ec2.internal,}" Jul 01 16:58:31.279025 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.278996 2580 policy_none.go:49] "None policy: Start" Jul 01 16:58:31.279025 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.279027 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 16:58:31.279106 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.279037 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 16:58:31.280339 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.279782 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.280621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.280020 2580 policy_none.go:47] "Start" Jul 01 16:58:31.281498 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.281483 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.281567 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.281506 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.281567 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.281520 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.336752 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.335729 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 16:58:31.336752 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.335768 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 16:58:31.336752 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.335786 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 16:58:31.336752 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.335821 2580 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 01 16:58:31.338256 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.338241 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:58:31.340602 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.340588 2580 manager.go:337] "Starting Device Plugin manager" Jul 01 16:58:31.340687 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.340623 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 16:58:31.340687 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.340637 2580 server.go:87] "Starting device plugin registration server" Jul 01 16:58:31.340899 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.340886 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 16:58:31.340988 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.340902 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 16:58:31.341343 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.341071 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 16:58:31.341343 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.341165 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 16:58:31.341343 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.341173 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 16:58:31.341645 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.341625 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 16:58:31.341732 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.341674 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.436836 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.436786 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal"] Jul 01 16:58:31.436913 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.436844 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.437633 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.437616 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.437719 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.437644 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.437719 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.437658 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.438974 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.438951 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.439093 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.439079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.439127 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.439112 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.440941 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.440922 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.441044 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.440971 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.441044 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.440985 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.441044 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.440994 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.441044 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.441023 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.441044 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.440987 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.441294 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.441039 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.441603 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.441574 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.441670 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.441617 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.441670 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.441632 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.441670 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.441655 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.442295 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.442278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.442378 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.442300 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:58:31.442864 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.442851 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:58:31.442925 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.442890 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:58:31.442925 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.442910 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:58:31.450296 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.450283 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.450361 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.450301 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-168.ec2.internal\": node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.476191 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.476060 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.476622 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.476599 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-168.ec2.internal\" not found" node="ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.483124 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.483109 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-168.ec2.internal\" not found" node="ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.550223 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.550202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bcd90a98582326580cdd920090dcc270-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal\" (UID: \"bcd90a98582326580cdd920090dcc270\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.550308 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.550225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd90a98582326580cdd920090dcc270-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal\" (UID: \"bcd90a98582326580cdd920090dcc270\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.550308 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.550244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ecf426856d647352e1707c5d6b53ca4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-168.ec2.internal\" (UID: \"0ecf426856d647352e1707c5d6b53ca4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.577141 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.577121 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.650971 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.650941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bcd90a98582326580cdd920090dcc270-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal\" (UID: \"bcd90a98582326580cdd920090dcc270\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.651042 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.650972 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bcd90a98582326580cdd920090dcc270-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal\" (UID: \"bcd90a98582326580cdd920090dcc270\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.651042 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.650988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd90a98582326580cdd920090dcc270-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal\" (UID: \"bcd90a98582326580cdd920090dcc270\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.651042 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.651006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ecf426856d647352e1707c5d6b53ca4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-168.ec2.internal\" (UID: \"0ecf426856d647352e1707c5d6b53ca4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.651042 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.651029 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ecf426856d647352e1707c5d6b53ca4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-168.ec2.internal\" (UID: \"0ecf426856d647352e1707c5d6b53ca4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.651167 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.651075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd90a98582326580cdd920090dcc270-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal\" (UID: \"bcd90a98582326580cdd920090dcc270\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.677735 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.677717 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.778193 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.778175 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.780291 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.780273 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.787137 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:31.787120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" Jul 01 16:58:31.878389 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.878363 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:31.978903 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:31.978881 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:32.080048 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:32.080029 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:32.182256 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:32.182229 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:32.184577 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.184561 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 16:58:32.184767 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.184749 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 16:58:32.265089 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.265057 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 16:53:31 +0000 UTC" deadline="2028-01-02 13:28:45.214424799 +0000 UTC" Jul 01 16:58:32.265089 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.265083 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13196h30m12.949345846s" Jul 01 16:58:32.283140 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:32.283121 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:32.383210 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:32.383161 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-168.ec2.internal\" not found" Jul 01 16:58:32.434322 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.434298 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:58:32.448166 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.448138 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" Jul 01 16:58:32.465452 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.465432 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 16:58:32.467655 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.467636 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" Jul 01 16:58:32.479858 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.479844 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 16:58:32.490961 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:32.490935 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecf426856d647352e1707c5d6b53ca4.slice/crio-8b443a88459fed93e018203195607ab1dbe191a8883d6d7b43b3694ab50527ec WatchSource:0}: Error finding container 8b443a88459fed93e018203195607ab1dbe191a8883d6d7b43b3694ab50527ec: Status 404 returned error can't find the container with id 8b443a88459fed93e018203195607ab1dbe191a8883d6d7b43b3694ab50527ec Jul 01 16:58:32.491416 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:32.491394 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd90a98582326580cdd920090dcc270.slice/crio-ab029e5eafe35d5c908bb499682b142c42757819e8f07c7f1a04a8380b1cdebb WatchSource:0}: Error finding container ab029e5eafe35d5c908bb499682b142c42757819e8f07c7f1a04a8380b1cdebb: Status 404 returned error can't find the container with id ab029e5eafe35d5c908bb499682b142c42757819e8f07c7f1a04a8380b1cdebb Jul 01 16:58:32.494954 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.494936 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:58:32.497628 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.497606 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 16:58:32.755978 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:32.755690 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:58:33.230052 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.230024 2580 apiserver.go:52] "Watching apiserver" Jul 01 16:58:33.240123 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.240094 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 16:58:33.240550 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.240528 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-46wnx","openshift-dns/node-resolver-tqhbs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal","openshift-multus/multus-qmkv2","openshift-multus/network-metrics-daemon-w4kj7","openshift-network-operator/iptables-alerter-p8knb","openshift-ovn-kubernetes/ovnkube-node-x9llh","kube-system/global-pull-secret-syncer-whqr9","kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn","openshift-image-registry/node-ca-cm2bx","openshift-multus/multus-additional-cni-plugins-2bzxl","openshift-network-diagnostics/network-check-target-dmnzt","kube-system/konnectivity-agent-m4cmr"] Jul 01 16:58:33.242529 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.242225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.243498 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.243473 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.245220 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.245189 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.246356 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.246076 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.246356 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.246132 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.246679 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.246660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.247930 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.247408 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 16:58:33.247930 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.247523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hq4hk\"" Jul 01 16:58:33.247930 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.247612 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 16:58:33.247930 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.247739 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 16:58:33.247930 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.247802 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kh6xp\"" Jul 01 16:58:33.248465 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.248433 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.248465 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.248453 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 16:58:33.248769 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.248750 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.248867 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.248825 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:33.248985 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.248966 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 16:58:33.249895 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.249876 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 16:58:33.250010 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.249902 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 16:58:33.250441 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.250284 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5tgvz\"" Jul 01 16:58:33.251450 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.250952 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.251450 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.251181 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 16:58:33.251450 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.251316 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.254423 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.254408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.254511 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.254474 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:33.255277 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.255104 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.256068 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.255708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.257375 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.257112 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.260565 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-system-cni-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.260662 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-systemd\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.260662 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-var-lib-kubelet\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.260761 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/825e4cdc-a7b9-489a-887e-570bb93439a2-tmp\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.260761 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-systemd-units\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.260761 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/159725ea-9649-42d8-88f3-a323c4ab22d4-cni-binary-copy\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.260900 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-daemon-config\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.260900 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-multus-certs\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.260900 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-slash\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.260900 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260850 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-kubelet\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.260900 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-hostroot\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.261171 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysctl-d\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.261171 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-var-lib-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.261171 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.260980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-env-overrides\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.261171 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.261011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-cni-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.262195 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-run-netns\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.262291 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-etc-kubernetes\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.262343 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94tzp\" (UniqueName: \"kubernetes.io/projected/825e4cdc-a7b9-489a-887e-570bb93439a2-kube-api-access-94tzp\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.262343 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6606f186-4ef5-4ab9-8adf-51b016350930-iptables-alerter-script\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.262442 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262379 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-ovnkube-config\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.262442 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-cni-bin\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.262442 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.262442 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262430 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.262674 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262614 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 16:58:33.262725 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262693 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 16:58:33.263009 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262993 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.263187 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.262435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-ovnkube-script-lib\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263187 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263112 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 16:58:33.263187 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-modprobe-d\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.263187 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-lib-modules\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.263187 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnkjq\" (UniqueName: \"kubernetes.io/projected/f13ee94a-dccb-417d-9067-ba4b8de586ef-kube-api-access-lnkjq\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-sys\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-tuned\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc7892fa-6fbd-4939-b1e4-55b84f5c6369-agent-certs\") pod \"konnectivity-agent-m4cmr\" (UID: \"dc7892fa-6fbd-4939-b1e4-55b84f5c6369\") " pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-etc-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263386 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-kubernetes\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263443 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-kubelet\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z59d\" (UniqueName: \"kubernetes.io/projected/6606f186-4ef5-4ab9-8adf-51b016350930-kube-api-access-7z59d\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263521 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc7892fa-6fbd-4939-b1e4-55b84f5c6369-konnectivity-ca\") pod \"konnectivity-agent-m4cmr\" (UID: \"dc7892fa-6fbd-4939-b1e4-55b84f5c6369\") " pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263531 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-cni-bin\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpd7v\" (UniqueName: \"kubernetes.io/projected/bd251530-5dc0-48cf-9351-4b69edd423bc-kube-api-access-hpd7v\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-socket-dir-parent\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-netns\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.263741 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-958dp\" (UniqueName: \"kubernetes.io/projected/159725ea-9649-42d8-88f3-a323c4ab22d4-kube-api-access-958dp\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysconfig\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-cnibin\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-os-release\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-cni-multus\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.263955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6606f186-4ef5-4ab9-8adf-51b016350930-host-slash\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-systemd\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-node-log\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264096 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-log-socket\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264154 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nqg6l\"" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-conf-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264174 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8mhb6\"" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysctl-conf\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-run\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264227 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.264302 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-k8s-cni-cncf-io\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.265139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-ovn\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.265139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-cni-netd\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.265139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-host\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.265139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd251530-5dc0-48cf-9351-4b69edd423bc-ovn-node-metrics-cert\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.265139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264542 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x4wf9\"" Jul 01 16:58:33.265139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.264935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.267340 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.267320 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:33.267426 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.267384 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:33.267426 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.267409 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.267514 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.267506 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.267645 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.267628 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 16:58:33.267705 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.267642 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wc49c\"" Jul 01 16:58:33.268425 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.268405 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.268676 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.268659 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 16:58:33.268792 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.268777 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 16:58:33.272357 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.272334 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k769t\"" Jul 01 16:58:33.273008 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.272993 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xxs79\"" Jul 01 16:58:33.273198 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.273185 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.273338 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.273323 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.273665 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.273647 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 16:58:33.313216 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.313198 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f6qb4" Jul 01 16:58:33.324013 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.323972 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f6qb4" Jul 01 16:58:33.341864 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.341814 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" event={"ID":"0ecf426856d647352e1707c5d6b53ca4","Type":"ContainerStarted","Data":"8b443a88459fed93e018203195607ab1dbe191a8883d6d7b43b3694ab50527ec"} Jul 01 16:58:33.343294 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.343267 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" event={"ID":"bcd90a98582326580cdd920090dcc270","Type":"ContainerStarted","Data":"ab029e5eafe35d5c908bb499682b142c42757819e8f07c7f1a04a8380b1cdebb"} Jul 01 16:58:33.350470 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.350452 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 16:58:33.365382 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-system-cni-dir\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.365548 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-host\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.365548 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd251530-5dc0-48cf-9351-4b69edd423bc-ovn-node-metrics-cert\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.365548 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb377-78e3-414b-bc82-96ad20197434-tmp-dir\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.365548 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6k67\" (UniqueName: \"kubernetes.io/projected/b6eeb377-78e3-414b-bc82-96ad20197434-kube-api-access-m6k67\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.365548 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-host\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.365742 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-system-cni-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.365742 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-system-cni-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.365742 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-systemd\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.365742 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-var-lib-kubelet\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.365742 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/825e4cdc-a7b9-489a-887e-570bb93439a2-tmp\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.365742 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-systemd-units\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.365956 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.365956 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.365956 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6eeb377-78e3-414b-bc82-96ad20197434-hosts-file\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.365956 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/159725ea-9649-42d8-88f3-a323c4ab22d4-cni-binary-copy\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.365956 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-daemon-config\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.365956 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.365914 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 16:58:33.366537 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366514 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/159725ea-9649-42d8-88f3-a323c4ab22d4-cni-binary-copy\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.366650 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366557 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-multus-certs\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.366650 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366523 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-daemon-config\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.366763 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366649 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-systemd-units\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.366763 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366660 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-systemd\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.366763 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366683 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-slash\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.366763 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-var-lib-kubelet\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.366763 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-slash\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.366763 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-dbus\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-kubelet\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-hostroot\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysctl-d\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-var-lib-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-env-overrides\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-sys-fs\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-cni-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.366982 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.366981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-run-netns\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-cni-binary-copy\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-etc-kubernetes\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94tzp\" (UniqueName: \"kubernetes.io/projected/825e4cdc-a7b9-489a-887e-570bb93439a2-kube-api-access-94tzp\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-var-lib-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6606f186-4ef5-4ab9-8adf-51b016350930-iptables-alerter-script\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-ovnkube-config\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74wk\" (UniqueName: \"kubernetes.io/projected/291e48c3-8b05-4b03-a22d-8daf83066d72-kube-api-access-n74wk\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-cni-bin\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-ovnkube-script-lib\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-os-release\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-modprobe-d\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-lib-modules\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367449 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.367615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-multus-certs\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-kubelet\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnkjq\" (UniqueName: \"kubernetes.io/projected/f13ee94a-dccb-417d-9067-ba4b8de586ef-kube-api-access-lnkjq\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367533 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-hostroot\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367656 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysctl-d\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367708 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-lib-modules\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-sys\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-tuned\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc7892fa-6fbd-4939-b1e4-55b84f5c6369-agent-certs\") pod \"konnectivity-agent-m4cmr\" (UID: \"dc7892fa-6fbd-4939-b1e4-55b84f5c6369\") " pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-etc-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-registration-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367905 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-kubernetes\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-kubelet\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-cnibin\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.367989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z59d\" (UniqueName: \"kubernetes.io/projected/6606f186-4ef5-4ab9-8adf-51b016350930-kube-api-access-7z59d\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.368486 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc7892fa-6fbd-4939-b1e4-55b84f5c6369-konnectivity-ca\") pod \"konnectivity-agent-m4cmr\" (UID: \"dc7892fa-6fbd-4939-b1e4-55b84f5c6369\") " pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368024 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-cni-bin\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpd7v\" (UniqueName: \"kubernetes.io/projected/bd251530-5dc0-48cf-9351-4b69edd423bc-kube-api-access-hpd7v\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-ovnkube-config\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368176 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-socket-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368176 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-ovnkube-script-lib\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-device-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-socket-dir-parent\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-netns\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368252 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-modprobe-d\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-958dp\" (UniqueName: \"kubernetes.io/projected/159725ea-9649-42d8-88f3-a323c4ab22d4-kube-api-access-958dp\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.368275 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368283 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysconfig\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-kubelet-config\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.368330 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:58:33.868314344 +0000 UTC m=+3.037329239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368363 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-kubelet\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.369328 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eaf2fa6c-6ef1-466b-a465-32c6339204ba-host\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-cnibin\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-sys\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-os-release\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-cni-multus\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6606f186-4ef5-4ab9-8adf-51b016350930-host-slash\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-systemd\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-systemd\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369049 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc7892fa-6fbd-4939-b1e4-55b84f5c6369-konnectivity-ca\") pod \"konnectivity-agent-m4cmr\" (UID: \"dc7892fa-6fbd-4939-b1e4-55b84f5c6369\") " pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369091 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-cni-bin\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369168 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-socket-dir-parent\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-netns\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369371 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6606f186-4ef5-4ab9-8adf-51b016350930-iptables-alerter-script\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-cni-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369528 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysconfig\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369567 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-cnibin\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369606 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-os-release\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370145 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369631 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-cni-multus\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369656 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6606f186-4ef5-4ab9-8adf-51b016350930-host-slash\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-var-lib-cni-bin\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-etc-kubernetes\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369738 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-run-netns\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.369981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd251530-5dc0-48cf-9351-4b69edd423bc-env-overrides\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370024 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-node-log\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.368346 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-kubernetes\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-etc-openvswitch\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370143 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-node-log\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370166 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-log-socket\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfrzg\" (UniqueName: \"kubernetes.io/projected/89842b5f-2f8e-4113-8dc6-27a9c81b4957-kube-api-access-lfrzg\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370237 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-log-socket\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370290 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-conf-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysctl-conf\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/825e4cdc-a7b9-489a-887e-570bb93439a2-tmp\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370337 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-multus-conf-dir\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.370906 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-run\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370414 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eaf2fa6c-6ef1-466b-a465-32c6339204ba-serviceca\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370468 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-sysctl-conf\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxq5j\" (UniqueName: \"kubernetes.io/projected/eaf2fa6c-6ef1-466b-a465-32c6339204ba-kube-api-access-cxq5j\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-k8s-cni-cncf-io\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370547 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/159725ea-9649-42d8-88f3-a323c4ab22d4-host-run-k8s-cni-cncf-io\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370556 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd251530-5dc0-48cf-9351-4b69edd423bc-ovn-node-metrics-cert\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-ovn\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-cni-netd\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370640 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-run-ovn\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370607 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/825e4cdc-a7b9-489a-887e-570bb93439a2-run\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370687 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd251530-5dc0-48cf-9351-4b69edd423bc-host-cni-netd\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.371631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.370689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.372387 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.372368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/825e4cdc-a7b9-489a-887e-570bb93439a2-etc-tuned\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.372675 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.372647 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc7892fa-6fbd-4939-b1e4-55b84f5c6369-agent-certs\") pod \"konnectivity-agent-m4cmr\" (UID: \"dc7892fa-6fbd-4939-b1e4-55b84f5c6369\") " pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.383635 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.383558 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnkjq\" (UniqueName: \"kubernetes.io/projected/f13ee94a-dccb-417d-9067-ba4b8de586ef-kube-api-access-lnkjq\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.384855 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.384835 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z59d\" (UniqueName: \"kubernetes.io/projected/6606f186-4ef5-4ab9-8adf-51b016350930-kube-api-access-7z59d\") pod \"iptables-alerter-p8knb\" (UID: \"6606f186-4ef5-4ab9-8adf-51b016350930\") " pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.385076 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.385057 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94tzp\" (UniqueName: \"kubernetes.io/projected/825e4cdc-a7b9-489a-887e-570bb93439a2-kube-api-access-94tzp\") pod \"tuned-46wnx\" (UID: \"825e4cdc-a7b9-489a-887e-570bb93439a2\") " pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.385257 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.385238 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-958dp\" (UniqueName: \"kubernetes.io/projected/159725ea-9649-42d8-88f3-a323c4ab22d4-kube-api-access-958dp\") pod \"multus-qmkv2\" (UID: \"159725ea-9649-42d8-88f3-a323c4ab22d4\") " pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.385300 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.385286 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpd7v\" (UniqueName: \"kubernetes.io/projected/bd251530-5dc0-48cf-9351-4b69edd423bc-kube-api-access-hpd7v\") pod \"ovnkube-node-x9llh\" (UID: \"bd251530-5dc0-48cf-9351-4b69edd423bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.471710 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-kubelet-config\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.471877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.471877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eaf2fa6c-6ef1-466b-a465-32c6339204ba-host\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.471877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfrzg\" (UniqueName: \"kubernetes.io/projected/89842b5f-2f8e-4113-8dc6-27a9c81b4957-kube-api-access-lfrzg\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.471877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471820 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-kubelet-config\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.471877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471834 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eaf2fa6c-6ef1-466b-a465-32c6339204ba-host\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.471877 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eaf2fa6c-6ef1-466b-a465-32c6339204ba-serviceca\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxq5j\" (UniqueName: \"kubernetes.io/projected/eaf2fa6c-6ef1-466b-a465-32c6339204ba-kube-api-access-cxq5j\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471937 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.471946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-system-cni-dir\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb377-78e3-414b-bc82-96ad20197434-tmp-dir\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472067 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6k67\" (UniqueName: \"kubernetes.io/projected/b6eeb377-78e3-414b-bc82-96ad20197434-kube-api-access-m6k67\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-system-cni-dir\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6eeb377-78e3-414b-bc82-96ad20197434-hosts-file\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-dbus\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.472250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-sys-fs\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472290 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-cni-binary-copy\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472317 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74wk\" (UniqueName: \"kubernetes.io/projected/291e48c3-8b05-4b03-a22d-8daf83066d72-kube-api-access-n74wk\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-os-release\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-registration-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472464 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb377-78e3-414b-bc82-96ad20197434-tmp-dir\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-cnibin\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472532 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-registration-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472473 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-socket-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472560 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-dbus\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472629 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-sys-fs\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472630 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-socket-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.472635 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:33.472963 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472648 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472664 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-cnibin\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-device-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.472699 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:33.972681274 +0000 UTC m=+3.141696174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472726 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/291e48c3-8b05-4b03-a22d-8daf83066d72-os-release\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472754 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89842b5f-2f8e-4113-8dc6-27a9c81b4957-device-dir\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6eeb377-78e3-414b-bc82-96ad20197434-hosts-file\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.472900 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eaf2fa6c-6ef1-466b-a465-32c6339204ba-serviceca\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.473621 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.473089 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/291e48c3-8b05-4b03-a22d-8daf83066d72-cni-binary-copy\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.491292 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.491214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74wk\" (UniqueName: \"kubernetes.io/projected/291e48c3-8b05-4b03-a22d-8daf83066d72-kube-api-access-n74wk\") pod \"multus-additional-cni-plugins-2bzxl\" (UID: \"291e48c3-8b05-4b03-a22d-8daf83066d72\") " pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.492064 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.492040 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:33.492064 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.492064 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:33.492223 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.492078 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:33.492223 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.492140 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:58:33.992127883 +0000 UTC m=+3.161142777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:33.494441 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.494415 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxq5j\" (UniqueName: \"kubernetes.io/projected/eaf2fa6c-6ef1-466b-a465-32c6339204ba-kube-api-access-cxq5j\") pod \"node-ca-cm2bx\" (UID: \"eaf2fa6c-6ef1-466b-a465-32c6339204ba\") " pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.498913 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.498888 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfrzg\" (UniqueName: \"kubernetes.io/projected/89842b5f-2f8e-4113-8dc6-27a9c81b4957-kube-api-access-lfrzg\") pod \"aws-ebs-csi-driver-node-vdbqn\" (UID: \"89842b5f-2f8e-4113-8dc6-27a9c81b4957\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.499820 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.499799 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6k67\" (UniqueName: \"kubernetes.io/projected/b6eeb377-78e3-414b-bc82-96ad20197434-kube-api-access-m6k67\") pod \"node-resolver-tqhbs\" (UID: \"b6eeb377-78e3-414b-bc82-96ad20197434\") " pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.559044 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.558998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p8knb" Jul 01 16:58:33.568021 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.567997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:33.577289 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.577260 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:33.584077 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.584058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qmkv2" Jul 01 16:58:33.592555 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.592486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46wnx" Jul 01 16:58:33.599385 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.599364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" Jul 01 16:58:33.607294 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.607271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cm2bx" Jul 01 16:58:33.615720 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.615699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" Jul 01 16:58:33.621805 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.621787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tqhbs" Jul 01 16:58:33.784481 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.784404 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:58:33.877241 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.877208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:33.877398 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.877353 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:33.877441 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.877405 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:58:34.87739126 +0000 UTC m=+4.046406153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:33.978279 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:33.978233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:33.978447 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.978425 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:33.978524 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:33.978512 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:34.978490605 +0000 UTC m=+4.147505519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:34.079001 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:34.078902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:34.079145 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.079059 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:34.079145 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.079079 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:34.079145 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.079091 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:34.079246 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.079152 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:58:35.079129526 +0000 UTC m=+4.248144421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:34.325939 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:34.325901 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:53:33 +0000 UTC" deadline="2028-02-15 22:41:27.527404806 +0000 UTC" Jul 01 16:58:34.325939 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:34.325933 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14261h42m53.201474782s" Jul 01 16:58:34.663313 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:34.663270 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159725ea_9649_42d8_88f3_a323c4ab22d4.slice/crio-4f582200584cd1e479e1e72b6389d8574708801a55fa543e7e2fea9a86c33f0d WatchSource:0}: Error finding container 4f582200584cd1e479e1e72b6389d8574708801a55fa543e7e2fea9a86c33f0d: Status 404 returned error can't find the container with id 4f582200584cd1e479e1e72b6389d8574708801a55fa543e7e2fea9a86c33f0d Jul 01 16:58:34.678674 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:34.678641 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6eeb377_78e3_414b_bc82_96ad20197434.slice/crio-73b797a424d485b9418aca69bf51b8c5fc9a7e24703561a73f792fb0a708e61f WatchSource:0}: Error finding container 73b797a424d485b9418aca69bf51b8c5fc9a7e24703561a73f792fb0a708e61f: Status 404 returned error can't find the container with id 73b797a424d485b9418aca69bf51b8c5fc9a7e24703561a73f792fb0a708e61f Jul 01 16:58:34.679426 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:34.679403 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf2fa6c_6ef1_466b_a465_32c6339204ba.slice/crio-5c20ddefaea8dc73ef570fff923721219694c55bad8cb3c125900ac4c2bbedb9 WatchSource:0}: Error finding container 5c20ddefaea8dc73ef570fff923721219694c55bad8cb3c125900ac4c2bbedb9: Status 404 returned error can't find the container with id 5c20ddefaea8dc73ef570fff923721219694c55bad8cb3c125900ac4c2bbedb9 Jul 01 16:58:34.681177 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:34.681130 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89842b5f_2f8e_4113_8dc6_27a9c81b4957.slice/crio-963a2e2fc95a351a7e40be3de9cb545d87874aa4873236041c3e918da57f52f7 WatchSource:0}: Error finding container 963a2e2fc95a351a7e40be3de9cb545d87874aa4873236041c3e918da57f52f7: Status 404 returned error can't find the container with id 963a2e2fc95a351a7e40be3de9cb545d87874aa4873236041c3e918da57f52f7 Jul 01 16:58:34.682512 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:34.682374 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd251530_5dc0_48cf_9351_4b69edd423bc.slice/crio-24a802951789f133749f4eec9c6280114b7243bfec7dc1bd334b23d2e1b8e6ee WatchSource:0}: Error finding container 24a802951789f133749f4eec9c6280114b7243bfec7dc1bd334b23d2e1b8e6ee: Status 404 returned error can't find the container with id 24a802951789f133749f4eec9c6280114b7243bfec7dc1bd334b23d2e1b8e6ee Jul 01 16:58:34.684227 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:58:34.684199 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6606f186_4ef5_4ab9_8adf_51b016350930.slice/crio-ddfff812e0e8be34fd2f335f0906f0093fa4060c55bcf0e51fdadf8c77f639de WatchSource:0}: Error finding container ddfff812e0e8be34fd2f335f0906f0093fa4060c55bcf0e51fdadf8c77f639de: Status 404 returned error can't find the container with id ddfff812e0e8be34fd2f335f0906f0093fa4060c55bcf0e51fdadf8c77f639de Jul 01 16:58:34.884523 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:34.884315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:34.884523 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.884506 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:34.884850 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.884586 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:58:36.884566435 +0000 UTC m=+6.053581329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:34.985401 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:34.985372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:34.985534 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.985515 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:34.985595 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:34.985580 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:36.985561396 +0000 UTC m=+6.154576316 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:35.087785 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.086119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:35.087785 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.086298 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:35.087785 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.086320 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:35.087785 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.086334 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:35.087785 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.086392 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:58:37.086373237 +0000 UTC m=+6.255388133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:35.326785 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.326535 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:53:33 +0000 UTC" deadline="2028-04-12 06:06:48.080172482 +0000 UTC" Jul 01 16:58:35.326785 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.326598 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15613h8m12.753578351s" Jul 01 16:58:35.338736 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.338687 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:35.338851 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.338831 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:35.339290 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.339266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:35.339418 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.339373 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:35.339502 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.339437 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:35.339565 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:35.339509 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:35.357815 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.357784 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" event={"ID":"0ecf426856d647352e1707c5d6b53ca4","Type":"ContainerStarted","Data":"f1ed04ed6e71ba32fa4dac88cb52e335c59abb26ecae1f0e7d48cbe026914f6f"} Jul 01 16:58:35.365603 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.365580 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcd90a98582326580cdd920090dcc270" containerID="e21281dc24851c2a5392d24b017e6a4d6d07ca5a9d232971028ec5441284de08" exitCode=0 Jul 01 16:58:35.366070 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.366028 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" event={"ID":"bcd90a98582326580cdd920090dcc270","Type":"ContainerDied","Data":"e21281dc24851c2a5392d24b017e6a4d6d07ca5a9d232971028ec5441284de08"} Jul 01 16:58:35.389027 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.387384 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerStarted","Data":"4f10aefe34736e83b0a13aa04a6a8bb092d3e7a9c331c4bbfd7f5834c3cc3b13"} Jul 01 16:58:35.396398 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.396368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46wnx" event={"ID":"825e4cdc-a7b9-489a-887e-570bb93439a2","Type":"ContainerStarted","Data":"3bc512c6bdca18130b937cd4e7b807dc7383c51b823374ebf6e6fbb5b7f1b891"} Jul 01 16:58:35.399380 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.399329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4cmr" event={"ID":"dc7892fa-6fbd-4939-b1e4-55b84f5c6369","Type":"ContainerStarted","Data":"e59277a1774283f93a86624885180ef6e1778eec854d39bc9ddbdec14dd0bce4"} Jul 01 16:58:35.402332 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.402198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p8knb" event={"ID":"6606f186-4ef5-4ab9-8adf-51b016350930","Type":"ContainerStarted","Data":"ddfff812e0e8be34fd2f335f0906f0093fa4060c55bcf0e51fdadf8c77f639de"} Jul 01 16:58:35.406191 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.406118 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cm2bx" event={"ID":"eaf2fa6c-6ef1-466b-a465-32c6339204ba","Type":"ContainerStarted","Data":"5c20ddefaea8dc73ef570fff923721219694c55bad8cb3c125900ac4c2bbedb9"} Jul 01 16:58:35.408137 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.407948 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-168.ec2.internal" podStartSLOduration=3.40791693 podStartE2EDuration="3.40791693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:35.385509982 +0000 UTC m=+4.554524898" watchObservedRunningTime="2026-07-01 16:58:35.40791693 +0000 UTC m=+4.576931850" Jul 01 16:58:35.408667 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.408564 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" event={"ID":"89842b5f-2f8e-4113-8dc6-27a9c81b4957","Type":"ContainerStarted","Data":"963a2e2fc95a351a7e40be3de9cb545d87874aa4873236041c3e918da57f52f7"} Jul 01 16:58:35.418447 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.418412 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"24a802951789f133749f4eec9c6280114b7243bfec7dc1bd334b23d2e1b8e6ee"} Jul 01 16:58:35.427775 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.427728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tqhbs" event={"ID":"b6eeb377-78e3-414b-bc82-96ad20197434","Type":"ContainerStarted","Data":"73b797a424d485b9418aca69bf51b8c5fc9a7e24703561a73f792fb0a708e61f"} Jul 01 16:58:35.437076 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:35.437047 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qmkv2" event={"ID":"159725ea-9649-42d8-88f3-a323c4ab22d4","Type":"ContainerStarted","Data":"4f582200584cd1e479e1e72b6389d8574708801a55fa543e7e2fea9a86c33f0d"} Jul 01 16:58:36.458150 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:36.458107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" event={"ID":"bcd90a98582326580cdd920090dcc270","Type":"ContainerStarted","Data":"6d4a4df6d7a566f902e6c4fad14195d039035978d8ddbfc1803ebcfc223d53ac"} Jul 01 16:58:36.903351 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:36.903279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:36.903543 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:36.903477 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:36.903543 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:36.903541 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:58:40.903522309 +0000 UTC m=+10.072537203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:37.004990 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:37.004334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:37.004990 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.004507 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:37.004990 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.004571 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:41.004552468 +0000 UTC m=+10.173567365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:37.105366 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:37.104694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:37.105366 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.104863 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:37.105366 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.104884 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:37.105366 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.104898 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:37.105366 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.104963 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:58:41.104942227 +0000 UTC m=+10.273957125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:37.337152 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:37.337098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:37.337356 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.337246 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:37.337744 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:37.337701 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:37.337881 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.337852 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:37.338149 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:37.338103 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:37.340691 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:37.338223 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:39.339178 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:39.339149 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:39.339614 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:39.339286 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:39.339677 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:39.339635 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:39.339748 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:39.339724 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:39.339804 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:39.339769 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:39.339854 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:39.339837 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:40.937241 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:40.937198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:40.937665 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:40.937315 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:40.937665 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:40.937375 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:58:48.937357669 +0000 UTC m=+18.106372569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:41.038431 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:41.038396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:41.038599 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.038534 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:41.038599 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.038588 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:49.038570992 +0000 UTC m=+18.207585889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:41.139218 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:41.139158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:41.139392 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.139363 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:41.139392 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.139390 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:41.139572 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.139403 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:41.139572 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.139461 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:58:49.139442001 +0000 UTC m=+18.308456897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:41.339507 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:41.339472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:41.339672 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.339562 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:41.339672 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:41.339632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:41.339762 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.339682 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:41.339762 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:41.339719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:41.339852 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:41.339764 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:43.337009 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:43.336978 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:43.337393 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:43.337011 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:43.337393 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:43.337027 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:43.337393 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:43.337123 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:43.337393 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:43.337215 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:43.337393 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:43.337296 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:45.336383 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:45.336351 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:45.336383 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:45.336378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:45.336865 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:45.336351 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:45.336865 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:45.336495 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:45.336865 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:45.336572 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:45.336865 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:45.336687 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:47.336360 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:47.336327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:47.336825 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:47.336463 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:47.336899 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:47.336859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:47.337007 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:47.336987 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:47.337078 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:47.337043 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:47.337183 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:47.337140 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:49.000198 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:49.000168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:49.000640 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.000285 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:49.000640 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.000345 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:59:05.000331273 +0000 UTC m=+34.169346180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:49.101130 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:49.101092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:49.101296 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.101257 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:49.101345 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.101339 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:05.101318755 +0000 UTC m=+34.270333658 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:49.202405 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:49.202369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:49.202558 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.202536 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:49.202558 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.202557 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:49.202645 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.202566 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:49.202645 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.202620 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:59:05.202606782 +0000 UTC m=+34.371621674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:49.339287 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:49.339216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:49.339287 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:49.339249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:49.339472 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.339327 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:49.339472 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:49.339378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:49.339571 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.339482 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:49.339615 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:49.339575 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:51.336631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:51.336601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:51.337087 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:51.336704 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:51.337087 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:51.336795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:51.337087 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:51.336950 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:51.337087 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:51.336995 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:51.337238 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:51.337100 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:53.336286 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:53.336248 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:53.336665 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:53.336290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:53.336665 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:53.336369 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:53.336665 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:53.336249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:53.336665 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:53.336469 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:53.336665 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:53.336558 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:53.497824 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:53.496200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4cmr" event={"ID":"dc7892fa-6fbd-4939-b1e4-55b84f5c6369","Type":"ContainerStarted","Data":"c59351a44b9e812807f6f35dffe87fd354760e0bd5e37128e46b6dea65b801c1"} Jul 01 16:58:53.517287 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:53.517246 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-168.ec2.internal" podStartSLOduration=21.517214767 podStartE2EDuration="21.517214767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:36.477145216 +0000 UTC m=+5.646160135" watchObservedRunningTime="2026-07-01 16:58:53.517214767 +0000 UTC m=+22.686229672" Jul 01 16:58:53.517810 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:53.517788 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m4cmr" podStartSLOduration=11.946492265 podStartE2EDuration="22.51777918s" totalImagesPullingTime="10.571286915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:53.517357358 +0000 UTC m=+22.686372253" watchObservedRunningTime="2026-07-01 16:58:53.51777918 +0000 UTC m=+22.686794121" Jul 01 16:58:54.499469 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.499269 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46wnx" event={"ID":"825e4cdc-a7b9-489a-887e-570bb93439a2","Type":"ContainerStarted","Data":"1c6bacdd386acc45a66652bf4a4cbb59bda8ce4f015ccf50cdf42576728d9b23"} Jul 01 16:58:54.500817 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.500783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cm2bx" event={"ID":"eaf2fa6c-6ef1-466b-a465-32c6339204ba","Type":"ContainerStarted","Data":"86c4f2efac90d8a97bb4477e947a96fc10cf3e8b3d605176edf6049bdb308c87"} Jul 01 16:58:54.502073 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.502052 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" event={"ID":"89842b5f-2f8e-4113-8dc6-27a9c81b4957","Type":"ContainerStarted","Data":"c5b14b1ee37cdc4ba098cffb31ecc0bb4c712c8e46b099690288ef36000ad2de"} Jul 01 16:58:54.504717 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.504698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 16:58:54.505074 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505049 2580 generic.go:356] "Generic (PLEG): container finished" podID="bd251530-5dc0-48cf-9351-4b69edd423bc" containerID="b5a6d7f23ff8a143bcadbf8ddff286845a1b37304b31cf5c83386aede76d264a" exitCode=1 Jul 01 16:58:54.505146 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"0f7e3260bb2128566e6e745a80ec5e228cc979b57efc948a72581138f077b96f"} Jul 01 16:58:54.505216 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505201 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"09b0bf15c21e2fd8ef6fdaefb4bce8968d44bb49772544ae42226e89605c715a"} Jul 01 16:58:54.505267 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505243 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"a00abc7fc060fcea73f6b5c847ee74638bc2f63badefc0f512b0a59e2336a934"} Jul 01 16:58:54.505308 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"78358f634cca79d37c16b5776bd8f5400519c3f6adfb66bcefee0464cfc3bcf1"} Jul 01 16:58:54.505308 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerDied","Data":"b5a6d7f23ff8a143bcadbf8ddff286845a1b37304b31cf5c83386aede76d264a"} Jul 01 16:58:54.505404 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.505326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"9858e069a7f8aeec2b2784d73ff7ed6541182666060216f4c18a0d2814a7481c"} Jul 01 16:58:54.506408 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.506391 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tqhbs" event={"ID":"b6eeb377-78e3-414b-bc82-96ad20197434","Type":"ContainerStarted","Data":"0325ff1dfb8f870e3a6ce88ced00f9f12baf09218f1d0259dbc321b5c5bf0bd4"} Jul 01 16:58:54.507690 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.507664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qmkv2" event={"ID":"159725ea-9649-42d8-88f3-a323c4ab22d4","Type":"ContainerStarted","Data":"4cf91d12c9a096268d5c9599470223d50ad7598533ebff218c4071d5d9651969"} Jul 01 16:58:54.509011 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.508990 2580 generic.go:356] "Generic (PLEG): container finished" podID="291e48c3-8b05-4b03-a22d-8daf83066d72" containerID="1a898c16df09006bfdbfc76f7bd99b40a6005ce5c9bf52b92873ae15d4ee76b1" exitCode=0 Jul 01 16:58:54.509093 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.509039 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerDied","Data":"1a898c16df09006bfdbfc76f7bd99b40a6005ce5c9bf52b92873ae15d4ee76b1"} Jul 01 16:58:54.520558 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.520525 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-46wnx" podStartSLOduration=3.888742299 podStartE2EDuration="22.520514731s" totalImagesPullingTime="18.631772432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:54.520165843 +0000 UTC m=+23.689180756" watchObservedRunningTime="2026-07-01 16:58:54.520514731 +0000 UTC m=+23.689529684" Jul 01 16:58:54.543014 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.542979 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qmkv2" podStartSLOduration=3.82391674 podStartE2EDuration="22.542951334s" totalImagesPullingTime="18.719034594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:54.542878778 +0000 UTC m=+23.711893691" watchObservedRunningTime="2026-07-01 16:58:54.542951334 +0000 UTC m=+23.711966247" Jul 01 16:58:54.558066 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.558034 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cm2bx" podStartSLOduration=3.919587205 podStartE2EDuration="22.558023094s" totalImagesPullingTime="18.638435889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:54.557840086 +0000 UTC m=+23.726854999" watchObservedRunningTime="2026-07-01 16:58:54.558023094 +0000 UTC m=+23.727038008" Jul 01 16:58:54.608976 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.608921 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tqhbs" podStartSLOduration=3.969371011 podStartE2EDuration="22.608905809s" totalImagesPullingTime="18.639534798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:54.608775137 +0000 UTC m=+23.777790069" watchObservedRunningTime="2026-07-01 16:58:54.608905809 +0000 UTC m=+23.777920723" Jul 01 16:58:54.677136 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:54.677107 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 16:58:55.337521 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.337484 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:55.337697 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:55.337652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:55.337697 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.337663 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:55.337869 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:55.337849 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:55.337928 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.337918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:55.338166 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:55.338147 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:55.354833 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.354745 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T16:58:54.677123801Z","UUID":"f12099ba-3b86-4d50-9adc-39144483e83f","Handler":null,"Name":"","Endpoint":""} Jul 01 16:58:55.356631 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.356608 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 16:58:55.356721 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.356654 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 16:58:55.514332 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.514254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" event={"ID":"89842b5f-2f8e-4113-8dc6-27a9c81b4957","Type":"ContainerStarted","Data":"f63c01efb60dc7275669fda86a100b36a80df2f1b76f0e8030297171c8d7badd"} Jul 01 16:58:55.518946 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.518891 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p8knb" event={"ID":"6606f186-4ef5-4ab9-8adf-51b016350930","Type":"ContainerStarted","Data":"7c907fa83ac392dc312312fbc04b02fabeb279cd7642c02e3ac75e5c0c06529e"} Jul 01 16:58:55.537926 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.537879 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p8knb" podStartSLOduration=5.904417834 podStartE2EDuration="24.537866348s" totalImagesPullingTime="18.633448514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:55.537764106 +0000 UTC m=+24.706779020" watchObservedRunningTime="2026-07-01 16:58:55.537866348 +0000 UTC m=+24.706881261" Jul 01 16:58:55.669420 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:55.669340 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:56.457151 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.456890 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:56.457584 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.457561 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:56.522762 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.522725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" event={"ID":"89842b5f-2f8e-4113-8dc6-27a9c81b4957","Type":"ContainerStarted","Data":"0c8df58a2f9d43737ef6b36d68fedc8d3b868acc0e7d839946faf6bf587e27eb"} Jul 01 16:58:56.526540 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.526522 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 16:58:56.527014 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.526987 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"d907e110e45644fd08ae7a4e4a981fede59831b34b48db6ab754048e0ce26370"} Jul 01 16:58:56.527720 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.527697 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m4cmr" Jul 01 16:58:56.545607 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:56.545560 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdbqn" podStartSLOduration=3.854886503 podStartE2EDuration="24.54554274s" totalImagesPullingTime="20.690656237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:56.545252087 +0000 UTC m=+25.714267000" watchObservedRunningTime="2026-07-01 16:58:56.54554274 +0000 UTC m=+25.714557653" Jul 01 16:58:57.336849 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:57.336817 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:57.337063 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:57.336860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:57.337063 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:57.336902 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:57.337063 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:57.337015 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:57.337226 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:57.337082 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:57.337226 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:57.337165 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:59.337171 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.336934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:58:59.337847 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.336934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:58:59.337847 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:59.337228 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:58:59.337847 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:59.337283 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:58:59.337847 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.337009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:58:59.337847 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:58:59.337369 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:58:59.536341 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.536318 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 16:58:59.536721 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.536698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"4fa7ac1e4c967302ecd6524fcd4b4eaff7933c8dd9b8a4adb8a010684cac8689"} Jul 01 16:58:59.536952 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.536936 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:59.537067 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.537002 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:58:59.537143 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.537126 2580 scope.go:117] "RemoveContainer" containerID="b5a6d7f23ff8a143bcadbf8ddff286845a1b37304b31cf5c83386aede76d264a" Jul 01 16:58:59.538669 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.538647 2580 generic.go:356] "Generic (PLEG): container finished" podID="291e48c3-8b05-4b03-a22d-8daf83066d72" containerID="45bc015a5c58b9abc3a3d14a993d04c7ecc0308ee954553d6374de7d58da0645" exitCode=0 Jul 01 16:58:59.538777 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.538684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerDied","Data":"45bc015a5c58b9abc3a3d14a993d04c7ecc0308ee954553d6374de7d58da0645"} Jul 01 16:58:59.554015 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:58:59.553994 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:59:00.545689 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.545666 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 16:59:00.546089 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.546060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" event={"ID":"bd251530-5dc0-48cf-9351-4b69edd423bc","Type":"ContainerStarted","Data":"5deb5ad36f43c5cfeb88865014f34b6ef8cb22c871448abaeb8d5c611b34d8d6"} Jul 01 16:59:00.546326 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.546304 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:59:00.561681 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.561656 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:59:00.581061 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.581016 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" podStartSLOduration=10.907287114 podStartE2EDuration="29.581002127s" totalImagesPullingTime="18.673715013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:00.580507382 +0000 UTC m=+29.749522307" watchObservedRunningTime="2026-07-01 16:59:00.581002127 +0000 UTC m=+29.750017041" Jul 01 16:59:00.910804 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.910592 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-whqr9"] Jul 01 16:59:00.910929 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.910845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:00.911019 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:00.910925 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:59:00.914388 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.914364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dmnzt"] Jul 01 16:59:00.914500 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.914447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:00.914552 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:00.914518 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:59:00.915081 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.915063 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4kj7"] Jul 01 16:59:00.915183 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:00.915145 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:00.915234 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:00.915218 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:59:01.550383 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:01.550347 2580 generic.go:356] "Generic (PLEG): container finished" podID="291e48c3-8b05-4b03-a22d-8daf83066d72" containerID="cd0485b703cbf76adeec7c82ba0ffc1983e0ce979086cca2b65af6d515c38d28" exitCode=0 Jul 01 16:59:01.550860 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:01.550431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerDied","Data":"cd0485b703cbf76adeec7c82ba0ffc1983e0ce979086cca2b65af6d515c38d28"} Jul 01 16:59:02.336207 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:02.336182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:02.336306 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:02.336289 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:59:02.336367 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:02.336347 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:02.336489 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:02.336468 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:59:02.554439 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:02.554409 2580 generic.go:356] "Generic (PLEG): container finished" podID="291e48c3-8b05-4b03-a22d-8daf83066d72" containerID="0fd08b391254bc62c5bd60829d3ddf02dd78aca898c3b6412ca60f8fae0d6e8d" exitCode=0 Jul 01 16:59:02.554911 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:02.554501 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerDied","Data":"0fd08b391254bc62c5bd60829d3ddf02dd78aca898c3b6412ca60f8fae0d6e8d"} Jul 01 16:59:03.336641 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:03.336615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:03.336807 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:03.336727 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:59:04.336796 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:04.336761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:04.337609 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:04.336774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:04.337609 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:04.336883 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:59:04.337609 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:04.336975 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:59:05.021190 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:05.021160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:05.021325 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.021298 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:59:05.021375 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.021365 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 16:59:37.021348317 +0000 UTC m=+66.190363212 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:59:05.121853 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:05.121775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:05.122023 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.121913 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:59:05.122023 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.121985 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret podName:f9b9fb6e-3231-44c6-8960-0ef674cd11d9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:37.121952881 +0000 UTC m=+66.290967773 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret") pod "global-pull-secret-syncer-whqr9" (UID: "f9b9fb6e-3231-44c6-8960-0ef674cd11d9") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:59:05.222386 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:05.222351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:05.222559 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.222525 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:59:05.222559 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.222541 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:59:05.222559 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.222550 2580 projected.go:196] Error preparing data for projected volume kube-api-access-rbrhc for pod openshift-network-diagnostics/network-check-target-dmnzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:59:05.222688 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.222609 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc podName:e4be5497-0027-4b62-972f-b355baab883f nodeName:}" failed. No retries permitted until 2026-07-01 16:59:37.222595105 +0000 UTC m=+66.391610015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rbrhc" (UniqueName: "kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc") pod "network-check-target-dmnzt" (UID: "e4be5497-0027-4b62-972f-b355baab883f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:59:05.336121 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:05.336091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:05.336286 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:05.336222 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmnzt" podUID="e4be5497-0027-4b62-972f-b355baab883f" Jul 01 16:59:06.336186 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.336116 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:06.336682 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.336126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:06.336682 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:06.336221 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-whqr9" podUID="f9b9fb6e-3231-44c6-8960-0ef674cd11d9" Jul 01 16:59:06.336682 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:06.336306 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4kj7" podUID="f13ee94a-dccb-417d-9067-ba4b8de586ef" Jul 01 16:59:06.593654 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.593592 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-168.ec2.internal" event="NodeReady" Jul 01 16:59:06.593782 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.593694 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 16:59:06.654157 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.654125 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mb9bv"] Jul 01 16:59:06.669164 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.669135 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8q8qz"] Jul 01 16:59:06.669301 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.669177 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.672502 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.672474 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7mnn7\"" Jul 01 16:59:06.672639 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.672621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 16:59:06.672719 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.672696 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 16:59:06.672761 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.672623 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 16:59:06.683339 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.683321 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mb9bv"] Jul 01 16:59:06.683427 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.683350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8q8qz"] Jul 01 16:59:06.683427 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.683418 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.686336 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.686321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wm6rt\"" Jul 01 16:59:06.686792 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.686778 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 16:59:06.687098 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.687085 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 16:59:06.834851 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.834821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-896zc\" (UniqueName: \"kubernetes.io/projected/1dbea86d-14fb-4a19-bbd8-50ff3cef1006-kube-api-access-896zc\") pod \"ingress-canary-mb9bv\" (UID: \"1dbea86d-14fb-4a19-bbd8-50ff3cef1006\") " pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.834851 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.834852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-config-volume\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.835032 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.834897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dbea86d-14fb-4a19-bbd8-50ff3cef1006-cert\") pod \"ingress-canary-mb9bv\" (UID: \"1dbea86d-14fb-4a19-bbd8-50ff3cef1006\") " pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.835032 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.834947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-tmp-dir\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.835032 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.835003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-944ss\" (UniqueName: \"kubernetes.io/projected/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-kube-api-access-944ss\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.835032 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.835022 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.936512 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.936484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-896zc\" (UniqueName: \"kubernetes.io/projected/1dbea86d-14fb-4a19-bbd8-50ff3cef1006-kube-api-access-896zc\") pod \"ingress-canary-mb9bv\" (UID: \"1dbea86d-14fb-4a19-bbd8-50ff3cef1006\") " pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.936615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.936536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-config-volume\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.936615 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.936609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dbea86d-14fb-4a19-bbd8-50ff3cef1006-cert\") pod \"ingress-canary-mb9bv\" (UID: \"1dbea86d-14fb-4a19-bbd8-50ff3cef1006\") " pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.936717 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.936641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-tmp-dir\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.936926 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.936907 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-tmp-dir\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.938406 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.937730 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-config-volume\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.938509 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.938446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-944ss\" (UniqueName: \"kubernetes.io/projected/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-kube-api-access-944ss\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.938565 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.938512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.938696 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:06.938674 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:06.938805 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:06.938752 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:07.438734538 +0000 UTC m=+36.607749447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:06.946623 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.946596 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-896zc\" (UniqueName: \"kubernetes.io/projected/1dbea86d-14fb-4a19-bbd8-50ff3cef1006-kube-api-access-896zc\") pod \"ingress-canary-mb9bv\" (UID: \"1dbea86d-14fb-4a19-bbd8-50ff3cef1006\") " pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.948423 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.948394 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1dbea86d-14fb-4a19-bbd8-50ff3cef1006-cert\") pod \"ingress-canary-mb9bv\" (UID: \"1dbea86d-14fb-4a19-bbd8-50ff3cef1006\") " pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:06.949151 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.949128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-944ss\" (UniqueName: \"kubernetes.io/projected/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-kube-api-access-944ss\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:06.980612 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:06.980594 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mb9bv" Jul 01 16:59:07.179681 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.179476 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mb9bv"] Jul 01 16:59:07.183000 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:07.182950 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dbea86d_14fb_4a19_bbd8_50ff3cef1006.slice/crio-182ea66e06c0039c78ca1e0d4881ba81f384485f021e576c1f24828c126da8a2 WatchSource:0}: Error finding container 182ea66e06c0039c78ca1e0d4881ba81f384485f021e576c1f24828c126da8a2: Status 404 returned error can't find the container with id 182ea66e06c0039c78ca1e0d4881ba81f384485f021e576c1f24828c126da8a2 Jul 01 16:59:07.336335 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.336312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:07.339195 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.339174 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 16:59:07.339310 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.339221 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6lq7p\"" Jul 01 16:59:07.339378 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.339344 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 16:59:07.442709 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.442637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:07.442823 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:07.442749 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:07.442823 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:07.442798 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:08.442783618 +0000 UTC m=+37.611798510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:07.567025 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.566998 2580 generic.go:356] "Generic (PLEG): container finished" podID="291e48c3-8b05-4b03-a22d-8daf83066d72" containerID="479293b531e52fd2b020fe8f9f0e40427428acf8f0e5db721a99b7e7a9972c68" exitCode=0 Jul 01 16:59:07.567176 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.567094 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerDied","Data":"479293b531e52fd2b020fe8f9f0e40427428acf8f0e5db721a99b7e7a9972c68"} Jul 01 16:59:07.568168 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:07.568140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mb9bv" event={"ID":"1dbea86d-14fb-4a19-bbd8-50ff3cef1006","Type":"ContainerStarted","Data":"182ea66e06c0039c78ca1e0d4881ba81f384485f021e576c1f24828c126da8a2"} Jul 01 16:59:08.336343 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.336309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:08.336789 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.336498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:08.339131 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.339106 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 16:59:08.339266 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.339165 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6c472\"" Jul 01 16:59:08.340123 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.340105 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 16:59:08.450858 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.450832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:08.451029 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:08.451013 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:08.451095 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:08.451083 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:10.451063852 +0000 UTC m=+39.620078749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:08.574774 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.574734 2580 generic.go:356] "Generic (PLEG): container finished" podID="291e48c3-8b05-4b03-a22d-8daf83066d72" containerID="a200e1613303573d1262dcb70186f5cccc8ca53b1547a4cfcc96ad34aa858619" exitCode=0 Jul 01 16:59:08.574944 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:08.574794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerDied","Data":"a200e1613303573d1262dcb70186f5cccc8ca53b1547a4cfcc96ad34aa858619"} Jul 01 16:59:09.583380 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:09.583152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" event={"ID":"291e48c3-8b05-4b03-a22d-8daf83066d72","Type":"ContainerStarted","Data":"8437b4c98edb571f87b0cdcdf4949b68a94ae08942c275c7b5a78f4ebdf08a39"} Jul 01 16:59:09.618746 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:09.618704 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2bzxl" podStartSLOduration=8.551804417 podStartE2EDuration="37.618689976s" totalImagesPullingTime="29.066885559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:09.618561607 +0000 UTC m=+38.787576520" watchObservedRunningTime="2026-07-01 16:59:09.618689976 +0000 UTC m=+38.787704889" Jul 01 16:59:10.464237 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:10.464199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:10.464396 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:10.464336 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:10.464396 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:10.464395 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:14.464381731 +0000 UTC m=+43.633396623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:10.586683 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:10.586651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mb9bv" event={"ID":"1dbea86d-14fb-4a19-bbd8-50ff3cef1006","Type":"ContainerStarted","Data":"661f9e84b5966e60b508186a008244106704f4a30663cdef7cf93a75e629e3a2"} Jul 01 16:59:14.490675 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:14.490621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:14.491170 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:14.490750 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:14.491170 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:14.490807 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:22.490793155 +0000 UTC m=+51.659808048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:22.540623 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:22.540584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:22.541087 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:22.540718 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:22.541087 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:22.540776 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:38.540761867 +0000 UTC m=+67.709776758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:32.565104 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:32.565076 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9llh" Jul 01 16:59:32.599573 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:32.599532 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mb9bv" podStartSLOduration=24.168062799 podStartE2EDuration="26.599518769s" totalImagesPullingTime="2.43145597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:10.626663299 +0000 UTC m=+39.795678211" watchObservedRunningTime="2026-07-01 16:59:32.599518769 +0000 UTC m=+61.768533675" Jul 01 16:59:37.039711 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.039675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 16:59:37.042330 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.042312 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 16:59:37.050527 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:37.050508 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 16:59:37.050602 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:37.050564 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs podName:f13ee94a-dccb-417d-9067-ba4b8de586ef nodeName:}" failed. No retries permitted until 2026-07-01 17:00:41.050549407 +0000 UTC m=+130.219564299 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs") pod "network-metrics-daemon-w4kj7" (UID: "f13ee94a-dccb-417d-9067-ba4b8de586ef") : secret "metrics-daemon-secret" not found Jul 01 16:59:37.140647 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.140627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:37.143523 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.143507 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 16:59:37.155563 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.155541 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9b9fb6e-3231-44c6-8960-0ef674cd11d9-original-pull-secret\") pod \"global-pull-secret-syncer-whqr9\" (UID: \"f9b9fb6e-3231-44c6-8960-0ef674cd11d9\") " pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:37.241601 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.241583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:37.244374 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.244358 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 16:59:37.254684 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.254665 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 16:59:37.265098 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.265075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbrhc\" (UniqueName: \"kubernetes.io/projected/e4be5497-0027-4b62-972f-b355baab883f-kube-api-access-rbrhc\") pod \"network-check-target-dmnzt\" (UID: \"e4be5497-0027-4b62-972f-b355baab883f\") " pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:37.347590 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.347537 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6lq7p\"" Jul 01 16:59:37.355632 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.355613 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:37.447553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.447529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-whqr9" Jul 01 16:59:37.500542 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.500516 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dmnzt"] Jul 01 16:59:37.503455 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:37.503430 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4be5497_0027_4b62_972f_b355baab883f.slice/crio-53107581466e6dd3e0b9e6cb66976d9747ed5afc60002d90ffc473314ee139a1 WatchSource:0}: Error finding container 53107581466e6dd3e0b9e6cb66976d9747ed5afc60002d90ffc473314ee139a1: Status 404 returned error can't find the container with id 53107581466e6dd3e0b9e6cb66976d9747ed5afc60002d90ffc473314ee139a1 Jul 01 16:59:37.589300 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.589269 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-whqr9"] Jul 01 16:59:37.592362 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:37.592335 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b9fb6e_3231_44c6_8960_0ef674cd11d9.slice/crio-652140e5b8d3b888102c2fecfde4629d1587f87ec116ec7f3869207f3c76040b WatchSource:0}: Error finding container 652140e5b8d3b888102c2fecfde4629d1587f87ec116ec7f3869207f3c76040b: Status 404 returned error can't find the container with id 652140e5b8d3b888102c2fecfde4629d1587f87ec116ec7f3869207f3c76040b Jul 01 16:59:37.646717 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.646683 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-whqr9" event={"ID":"f9b9fb6e-3231-44c6-8960-0ef674cd11d9","Type":"ContainerStarted","Data":"652140e5b8d3b888102c2fecfde4629d1587f87ec116ec7f3869207f3c76040b"} Jul 01 16:59:37.647819 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:37.647797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dmnzt" event={"ID":"e4be5497-0027-4b62-972f-b355baab883f","Type":"ContainerStarted","Data":"53107581466e6dd3e0b9e6cb66976d9747ed5afc60002d90ffc473314ee139a1"} Jul 01 16:59:38.550136 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:38.550097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 16:59:38.550661 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:38.550270 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:38.550661 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:38.550353 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls podName:e3a7cc17-7e6f-4a93-ba6b-f01760f73a48 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:10.550333265 +0000 UTC m=+99.719348160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls") pod "dns-default-8q8qz" (UID: "e3a7cc17-7e6f-4a93-ba6b-f01760f73a48") : secret "dns-default-metrics-tls" not found Jul 01 16:59:42.426908 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.426011 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-lxtr5"] Jul 01 16:59:42.428088 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.428062 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.440190 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.440165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 01 16:59:42.440712 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.440691 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 16:59:42.441024 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.441005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8s8gk\"" Jul 01 16:59:42.451511 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.451493 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 16:59:42.454265 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.454245 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 01 16:59:42.467477 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.467458 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 01 16:59:42.478664 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.478643 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq66m\" (UniqueName: \"kubernetes.io/projected/e4cde9da-cbdc-4018-8639-7656ff9a614f-kube-api-access-fq66m\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.478762 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.478695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4cde9da-cbdc-4018-8639-7656ff9a614f-serving-cert\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.478762 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.478749 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4cde9da-cbdc-4018-8639-7656ff9a614f-tmp\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.478855 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.478787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cde9da-cbdc-4018-8639-7656ff9a614f-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.478855 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.478832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e4cde9da-cbdc-4018-8639-7656ff9a614f-snapshots\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.478929 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.478857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cde9da-cbdc-4018-8639-7656ff9a614f-service-ca-bundle\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.489341 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.489314 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-lxtr5"] Jul 01 16:59:42.579373 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cde9da-cbdc-4018-8639-7656ff9a614f-service-ca-bundle\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579373 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq66m\" (UniqueName: \"kubernetes.io/projected/e4cde9da-cbdc-4018-8639-7656ff9a614f-kube-api-access-fq66m\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579503 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4cde9da-cbdc-4018-8639-7656ff9a614f-serving-cert\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579503 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4cde9da-cbdc-4018-8639-7656ff9a614f-tmp\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579503 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cde9da-cbdc-4018-8639-7656ff9a614f-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579503 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e4cde9da-cbdc-4018-8639-7656ff9a614f-snapshots\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579833 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579811 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4cde9da-cbdc-4018-8639-7656ff9a614f-tmp\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.579991 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.579949 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cde9da-cbdc-4018-8639-7656ff9a614f-service-ca-bundle\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.580074 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.580057 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e4cde9da-cbdc-4018-8639-7656ff9a614f-snapshots\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.580387 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.580366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4cde9da-cbdc-4018-8639-7656ff9a614f-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.581756 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.581735 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4cde9da-cbdc-4018-8639-7656ff9a614f-serving-cert\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.628098 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.628080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq66m\" (UniqueName: \"kubernetes.io/projected/e4cde9da-cbdc-4018-8639-7656ff9a614f-kube-api-access-fq66m\") pod \"insights-operator-5f67b55555-lxtr5\" (UID: \"e4cde9da-cbdc-4018-8639-7656ff9a614f\") " pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.663129 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.663103 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-whqr9" event={"ID":"f9b9fb6e-3231-44c6-8960-0ef674cd11d9","Type":"ContainerStarted","Data":"99f0ade5195c7f8af62929710454810209ef03cede373f42d563c3031ebdb511"} Jul 01 16:59:42.664559 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.664539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dmnzt" event={"ID":"e4be5497-0027-4b62-972f-b355baab883f","Type":"ContainerStarted","Data":"3208ad103970a29fac5a06210afd4c17ade557dba5ebf88210c69f7e9a34ec57"} Jul 01 16:59:42.664648 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.664587 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 16:59:42.677142 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.677125 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-mgmvf"] Jul 01 16:59:42.679800 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.679787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.684853 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.684837 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:59:42.685157 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.685138 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 01 16:59:42.685240 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.685140 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 01 16:59:42.685418 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.685403 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 01 16:59:42.686040 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.686020 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lfmk6\"" Jul 01 16:59:42.692407 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.692391 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 01 16:59:42.713703 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.713662 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-whqr9" podStartSLOduration=66.063134286 podStartE2EDuration="1m10.713650878s" totalImagesPullingTime="4.650516592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:42.708623364 +0000 UTC m=+71.877638277" watchObservedRunningTime="2026-07-01 16:59:42.713650878 +0000 UTC m=+71.882665790" Jul 01 16:59:42.714234 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.714220 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-mgmvf"] Jul 01 16:59:42.736444 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.736422 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" Jul 01 16:59:42.772884 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.772858 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-774d5bc7bd-87mt4"] Jul 01 16:59:42.774310 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.774271 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dmnzt" podStartSLOduration=66.035940342 podStartE2EDuration="1m10.774256767s" totalImagesPullingTime="4.738316425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:42.761387819 +0000 UTC m=+71.930402732" watchObservedRunningTime="2026-07-01 16:59:42.774256767 +0000 UTC m=+71.943271680" Jul 01 16:59:42.774744 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.774731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.780805 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.780651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/362fbfdc-5a59-4933-b0cf-384af6eba934-serving-cert\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.780805 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.780685 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/362fbfdc-5a59-4933-b0cf-384af6eba934-trusted-ca\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.780805 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.780718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/362fbfdc-5a59-4933-b0cf-384af6eba934-config\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.780805 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.780773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxcv2\" (UniqueName: \"kubernetes.io/projected/362fbfdc-5a59-4933-b0cf-384af6eba934-kube-api-access-bxcv2\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.785298 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.785269 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 16:59:42.786021 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.786002 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 16:59:42.793069 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.793041 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 16:59:42.793192 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.793179 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8gxrw\"" Jul 01 16:59:42.824757 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.824566 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 16:59:42.831675 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.831621 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-774d5bc7bd-87mt4"] Jul 01 16:59:42.868365 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.868345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-lxtr5"] Jul 01 16:59:42.871465 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:42.871434 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cde9da_cbdc_4018_8639_7656ff9a614f.slice/crio-6c14bb5e0079b217f55df6c60c8d8dd25fa3d230dcb67b81c446f1de80fa3ba6 WatchSource:0}: Error finding container 6c14bb5e0079b217f55df6c60c8d8dd25fa3d230dcb67b81c446f1de80fa3ba6: Status 404 returned error can't find the container with id 6c14bb5e0079b217f55df6c60c8d8dd25fa3d230dcb67b81c446f1de80fa3ba6 Jul 01 16:59:42.881748 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.881731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/362fbfdc-5a59-4933-b0cf-384af6eba934-config\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.881831 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.881759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.881831 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.881777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-trusted-ca\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.881831 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.881798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69739103-a77b-4018-a8f9-699da1459ee9-ca-trust-extracted\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.881831 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.881813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2kb6\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-kube-api-access-k2kb6\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.882015 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.881940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxcv2\" (UniqueName: \"kubernetes.io/projected/362fbfdc-5a59-4933-b0cf-384af6eba934-kube-api-access-bxcv2\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.882015 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-installation-pull-secrets\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.882107 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-image-registry-private-configuration\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.882107 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-bound-sa-token\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.882107 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-registry-certificates\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.882243 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/362fbfdc-5a59-4933-b0cf-384af6eba934-serving-cert\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.882243 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/362fbfdc-5a59-4933-b0cf-384af6eba934-trusted-ca\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.882445 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882425 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/362fbfdc-5a59-4933-b0cf-384af6eba934-config\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.883002 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.882986 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/362fbfdc-5a59-4933-b0cf-384af6eba934-trusted-ca\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.884362 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.884346 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/362fbfdc-5a59-4933-b0cf-384af6eba934-serving-cert\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.892916 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.892898 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxcv2\" (UniqueName: \"kubernetes.io/projected/362fbfdc-5a59-4933-b0cf-384af6eba934-kube-api-access-bxcv2\") pod \"console-operator-7fdbdf9c99-mgmvf\" (UID: \"362fbfdc-5a59-4933-b0cf-384af6eba934\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.983008 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.982991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983099 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-trusted-ca\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983099 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69739103-a77b-4018-a8f9-699da1459ee9-ca-trust-extracted\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983099 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kb6\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-kube-api-access-k2kb6\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983205 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:42.983130 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:59:42.983205 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:42.983145 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-774d5bc7bd-87mt4: secret "image-registry-tls" not found Jul 01 16:59:42.983205 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-installation-pull-secrets\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983205 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:42.983192 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls podName:69739103-a77b-4018-a8f9-699da1459ee9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:43.483179751 +0000 UTC m=+72.652194643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls") pod "image-registry-774d5bc7bd-87mt4" (UID: "69739103-a77b-4018-a8f9-699da1459ee9") : secret "image-registry-tls" not found Jul 01 16:59:42.983495 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-image-registry-private-configuration\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983495 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-bound-sa-token\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983495 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-registry-certificates\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983495 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69739103-a77b-4018-a8f9-699da1459ee9-ca-trust-extracted\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983790 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-registry-certificates\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.983919 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.983904 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-trusted-ca\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.985458 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.985437 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-image-registry-private-configuration\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.985747 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.985726 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-installation-pull-secrets\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:42.988637 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.988615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:42.998278 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:42.998257 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-bound-sa-token\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:43.001134 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:43.001112 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2kb6\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-kube-api-access-k2kb6\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:43.124275 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:43.124251 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-mgmvf"] Jul 01 16:59:43.126369 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:43.126341 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362fbfdc_5a59_4933_b0cf_384af6eba934.slice/crio-2fb7e018e1cebdae2087476b2ce31bc0dcee928ed4f0acee75ec48312724ca7e WatchSource:0}: Error finding container 2fb7e018e1cebdae2087476b2ce31bc0dcee928ed4f0acee75ec48312724ca7e: Status 404 returned error can't find the container with id 2fb7e018e1cebdae2087476b2ce31bc0dcee928ed4f0acee75ec48312724ca7e Jul 01 16:59:43.487672 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:43.487639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:43.488042 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:43.487789 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:59:43.488042 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:43.487810 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-774d5bc7bd-87mt4: secret "image-registry-tls" not found Jul 01 16:59:43.488042 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:43.487875 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls podName:69739103-a77b-4018-a8f9-699da1459ee9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:44.487854173 +0000 UTC m=+73.656869069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls") pod "image-registry-774d5bc7bd-87mt4" (UID: "69739103-a77b-4018-a8f9-699da1459ee9") : secret "image-registry-tls" not found Jul 01 16:59:43.667849 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:43.667816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" event={"ID":"e4cde9da-cbdc-4018-8639-7656ff9a614f","Type":"ContainerStarted","Data":"6c14bb5e0079b217f55df6c60c8d8dd25fa3d230dcb67b81c446f1de80fa3ba6"} Jul 01 16:59:43.668919 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:43.668886 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" event={"ID":"362fbfdc-5a59-4933-b0cf-384af6eba934","Type":"ContainerStarted","Data":"2fb7e018e1cebdae2087476b2ce31bc0dcee928ed4f0acee75ec48312724ca7e"} Jul 01 16:59:44.495674 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:44.495639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:44.496140 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:44.495781 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:59:44.496140 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:44.495794 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-774d5bc7bd-87mt4: secret "image-registry-tls" not found Jul 01 16:59:44.496140 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:44.495848 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls podName:69739103-a77b-4018-a8f9-699da1459ee9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:46.495827078 +0000 UTC m=+75.664841971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls") pod "image-registry-774d5bc7bd-87mt4" (UID: "69739103-a77b-4018-a8f9-699da1459ee9") : secret "image-registry-tls" not found Jul 01 16:59:45.675779 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:45.675693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" event={"ID":"e4cde9da-cbdc-4018-8639-7656ff9a614f","Type":"ContainerStarted","Data":"70b4cc3e9205e4c73b22a6e3aa242f1ca84aea121bfa2eee3bfa5c7444a5d912"} Jul 01 16:59:45.677415 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:45.677394 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/0.log" Jul 01 16:59:45.677513 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:45.677437 2580 generic.go:356] "Generic (PLEG): container finished" podID="362fbfdc-5a59-4933-b0cf-384af6eba934" containerID="f659b792f3e36cc92b2c6687b0d996f77773145a8af38641bc11825b26e8aecd" exitCode=255 Jul 01 16:59:45.677513 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:45.677502 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" event={"ID":"362fbfdc-5a59-4933-b0cf-384af6eba934","Type":"ContainerDied","Data":"f659b792f3e36cc92b2c6687b0d996f77773145a8af38641bc11825b26e8aecd"} Jul 01 16:59:45.677726 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:45.677713 2580 scope.go:117] "RemoveContainer" containerID="f659b792f3e36cc92b2c6687b0d996f77773145a8af38641bc11825b26e8aecd" Jul 01 16:59:45.697512 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:45.697479 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" podStartSLOduration=1.862023454 podStartE2EDuration="3.697463413s" totalImagesPullingTime="1.835439959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:45.696741707 +0000 UTC m=+74.865756622" watchObservedRunningTime="2026-07-01 16:59:45.697463413 +0000 UTC m=+74.866478325" Jul 01 16:59:46.164858 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.164830 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ssfv4"] Jul 01 16:59:46.167700 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.167685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.174180 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.174154 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-57ng4\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-57ng4\"" type="*v1.Secret" Jul 01 16:59:46.175187 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.175163 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-tls\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" Jul 01 16:59:46.175940 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.175917 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"insights-runtime-extractor-ssfv4\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" podUID="4566659d-c605-480c-8d0b-580e851b62f3" pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.176198 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.176179 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" Jul 01 16:59:46.208510 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.208488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v65d\" (UniqueName: \"kubernetes.io/projected/4566659d-c605-480c-8d0b-580e851b62f3-kube-api-access-4v65d\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.208600 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.208528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.208600 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.208548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4566659d-c605-480c-8d0b-580e851b62f3-data-volume\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.208674 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.208642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4566659d-c605-480c-8d0b-580e851b62f3-crio-socket\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.208716 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.208698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.220498 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.220474 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssfv4"] Jul 01 16:59:46.309379 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.309509 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v65d\" (UniqueName: \"kubernetes.io/projected/4566659d-c605-480c-8d0b-580e851b62f3-kube-api-access-4v65d\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.309509 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.309509 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4566659d-c605-480c-8d0b-580e851b62f3-data-volume\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.309656 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309634 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4566659d-c605-480c-8d0b-580e851b62f3-crio-socket\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.309764 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4566659d-c605-480c-8d0b-580e851b62f3-data-volume\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.309764 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.309752 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4566659d-c605-480c-8d0b-580e851b62f3-crio-socket\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.398716 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.398691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v65d\" (UniqueName: \"kubernetes.io/projected/4566659d-c605-480c-8d0b-580e851b62f3-kube-api-access-4v65d\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:46.511337 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.511314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:46.511457 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.511419 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:59:46.511457 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.511429 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-774d5bc7bd-87mt4: secret "image-registry-tls" not found Jul 01 16:59:46.511525 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.511477 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls podName:69739103-a77b-4018-a8f9-699da1459ee9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:50.511463959 +0000 UTC m=+79.680478851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls") pod "image-registry-774d5bc7bd-87mt4" (UID: "69739103-a77b-4018-a8f9-699da1459ee9") : secret "image-registry-tls" not found Jul 01 16:59:46.681812 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.681787 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 16:59:46.682208 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.682194 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/0.log" Jul 01 16:59:46.682248 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.682227 2580 generic.go:356] "Generic (PLEG): container finished" podID="362fbfdc-5a59-4933-b0cf-384af6eba934" containerID="1fbdebe5266815332490ce5c2a31d29a312b7acfae0ff07a6ab0967cb026834c" exitCode=255 Jul 01 16:59:46.682320 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.682306 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" event={"ID":"362fbfdc-5a59-4933-b0cf-384af6eba934","Type":"ContainerDied","Data":"1fbdebe5266815332490ce5c2a31d29a312b7acfae0ff07a6ab0967cb026834c"} Jul 01 16:59:46.682361 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.682350 2580 scope.go:117] "RemoveContainer" containerID="f659b792f3e36cc92b2c6687b0d996f77773145a8af38641bc11825b26e8aecd" Jul 01 16:59:46.682546 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:46.682520 2580 scope.go:117] "RemoveContainer" containerID="1fbdebe5266815332490ce5c2a31d29a312b7acfae0ff07a6ab0967cb026834c" Jul 01 16:59:46.682738 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:46.682715 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-mgmvf_openshift-console-operator(362fbfdc-5a59-4933-b0cf-384af6eba934)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" podUID="362fbfdc-5a59-4933-b0cf-384af6eba934" Jul 01 16:59:47.048553 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.048526 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 16:59:47.050562 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.050533 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:47.050649 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.050597 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:47.550582689 +0000 UTC m=+76.719597602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : configmap "kube-rbac-proxy" not found Jul 01 16:59:47.286608 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.286582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 16:59:47.290596 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.290582 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:47.290670 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.290640 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:47.790625701 +0000 UTC m=+76.959640593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:47.619905 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.619861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:47.620113 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.619996 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:47.620113 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.620060 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:48.620044974 +0000 UTC m=+77.789059868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : configmap "kube-rbac-proxy" not found Jul 01 16:59:47.632536 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.632513 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-57ng4\"" Jul 01 16:59:47.686975 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.686941 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 16:59:47.687358 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.687339 2580 scope.go:117] "RemoveContainer" containerID="1fbdebe5266815332490ce5c2a31d29a312b7acfae0ff07a6ab0967cb026834c" Jul 01 16:59:47.687524 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.687503 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-mgmvf_openshift-console-operator(362fbfdc-5a59-4933-b0cf-384af6eba934)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" podUID="362fbfdc-5a59-4933-b0cf-384af6eba934" Jul 01 16:59:47.821596 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:47.821565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:47.821750 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.821704 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:47.821788 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:47.821770 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:48.821753393 +0000 UTC m=+77.990768286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:48.208354 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:48.208325 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tqhbs_b6eeb377-78e3-414b-bc82-96ad20197434/dns-node-resolver/0.log" Jul 01 16:59:48.629451 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:48.629426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:48.629593 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:48.629532 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:48.629593 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:48.629578 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:50.629567243 +0000 UTC m=+79.798582135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : configmap "kube-rbac-proxy" not found Jul 01 16:59:48.831538 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:48.831510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:48.831842 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:48.831623 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:48.831842 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:48.831668 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:50.831655933 +0000 UTC m=+80.000670825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:48.999754 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:48.999730 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cm2bx_eaf2fa6c-6ef1-466b-a465-32c6339204ba/node-ca/0.log" Jul 01 16:59:49.169208 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.169177 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh"] Jul 01 16:59:49.174126 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.174102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" Jul 01 16:59:49.176784 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.176756 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lz2ns\"" Jul 01 16:59:49.177040 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.177025 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 01 16:59:49.177595 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.177582 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 01 16:59:49.191139 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.191114 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh"] Jul 01 16:59:49.234155 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.234131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dzgw\" (UniqueName: \"kubernetes.io/projected/d9f77bb1-30cd-4c03-b45e-026b3113360a-kube-api-access-4dzgw\") pod \"migrator-786f9fbc5-ctxzh\" (UID: \"d9f77bb1-30cd-4c03-b45e-026b3113360a\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" Jul 01 16:59:49.334497 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.334439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dzgw\" (UniqueName: \"kubernetes.io/projected/d9f77bb1-30cd-4c03-b45e-026b3113360a-kube-api-access-4dzgw\") pod \"migrator-786f9fbc5-ctxzh\" (UID: \"d9f77bb1-30cd-4c03-b45e-026b3113360a\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" Jul 01 16:59:49.343896 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.343875 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dzgw\" (UniqueName: \"kubernetes.io/projected/d9f77bb1-30cd-4c03-b45e-026b3113360a-kube-api-access-4dzgw\") pod \"migrator-786f9fbc5-ctxzh\" (UID: \"d9f77bb1-30cd-4c03-b45e-026b3113360a\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" Jul 01 16:59:49.482612 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.482590 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" Jul 01 16:59:49.615266 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.615201 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh"] Jul 01 16:59:49.618429 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:49.618403 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f77bb1_30cd_4c03_b45e_026b3113360a.slice/crio-5ebe6e22d7ecea9b655492ca10c18491d1be8681b4e0d77f60748eeed782ba0f WatchSource:0}: Error finding container 5ebe6e22d7ecea9b655492ca10c18491d1be8681b4e0d77f60748eeed782ba0f: Status 404 returned error can't find the container with id 5ebe6e22d7ecea9b655492ca10c18491d1be8681b4e0d77f60748eeed782ba0f Jul 01 16:59:49.693950 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.693923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" event={"ID":"d9f77bb1-30cd-4c03-b45e-026b3113360a","Type":"ContainerStarted","Data":"5ebe6e22d7ecea9b655492ca10c18491d1be8681b4e0d77f60748eeed782ba0f"} Jul 01 16:59:49.999509 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:49.999485 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mb9bv_1dbea86d-14fb-4a19-bbd8-50ff3cef1006/serve-healthcheck-canary/0.log" Jul 01 16:59:50.542905 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:50.542873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:50.543071 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.543033 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:59:50.543071 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.543048 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-774d5bc7bd-87mt4: secret "image-registry-tls" not found Jul 01 16:59:50.543182 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.543107 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls podName:69739103-a77b-4018-a8f9-699da1459ee9 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:58.543088681 +0000 UTC m=+87.712103594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls") pod "image-registry-774d5bc7bd-87mt4" (UID: "69739103-a77b-4018-a8f9-699da1459ee9") : secret "image-registry-tls" not found Jul 01 16:59:50.644167 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:50.644141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:50.644313 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.644225 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:50.644313 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.644293 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:54.644276542 +0000 UTC m=+83.813291436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : configmap "kube-rbac-proxy" not found Jul 01 16:59:50.845266 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:50.845187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:50.845406 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.845352 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:50.845454 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:50.845424 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:54.845409599 +0000 UTC m=+84.014424491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:51.703704 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:51.703584 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" event={"ID":"d9f77bb1-30cd-4c03-b45e-026b3113360a","Type":"ContainerStarted","Data":"f1b9a47136c252e8c63cef656ac740e13ea385d7ded601b2994d65a3db7ad86b"} Jul 01 16:59:51.703704 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:51.703639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" event={"ID":"d9f77bb1-30cd-4c03-b45e-026b3113360a","Type":"ContainerStarted","Data":"4bddf9388d356f56ab2aaecffa6b4e52e1b90288c01d4d2f576e0ccc496cf525"} Jul 01 16:59:51.737268 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:51.735825 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-ctxzh" podStartSLOduration=1.3199095220000001 podStartE2EDuration="2.735806335s" totalImagesPullingTime="1.415896813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:51.733534855 +0000 UTC m=+80.902549780" watchObservedRunningTime="2026-07-01 16:59:51.735806335 +0000 UTC m=+80.904821251" Jul 01 16:59:52.989188 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:52.989154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:52.989641 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:52.989244 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 16:59:52.989707 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:52.989640 2580 scope.go:117] "RemoveContainer" containerID="1fbdebe5266815332490ce5c2a31d29a312b7acfae0ff07a6ab0967cb026834c" Jul 01 16:59:52.989841 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:52.989818 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-mgmvf_openshift-console-operator(362fbfdc-5a59-4933-b0cf-384af6eba934)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" podUID="362fbfdc-5a59-4933-b0cf-384af6eba934" Jul 01 16:59:54.670884 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:54.670847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:54.671273 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:54.670949 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:54.671273 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:54.671023 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:02.671006599 +0000 UTC m=+91.840021494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : configmap "kube-rbac-proxy" not found Jul 01 16:59:54.873159 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:54.873129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 16:59:54.873319 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:54.873262 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:54.873319 ip-10-0-140-168 kubenswrapper[2580]: E0701 16:59:54.873317 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls podName:4566659d-c605-480c-8d0b-580e851b62f3 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:02.873300281 +0000 UTC m=+92.042315175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssfv4" (UID: "4566659d-c605-480c-8d0b-580e851b62f3") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:58.602358 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:58.602323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:58.604698 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:58.604678 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"image-registry-774d5bc7bd-87mt4\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:58.686065 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:58.686038 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:58.835250 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:58.835228 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-774d5bc7bd-87mt4"] Jul 01 16:59:58.837080 ip-10-0-140-168 kubenswrapper[2580]: W0701 16:59:58.837055 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69739103_a77b_4018_a8f9_699da1459ee9.slice/crio-5e954479fe879c96dad899924e26912e19a1dc76078832f84305e8d5342f3445 WatchSource:0}: Error finding container 5e954479fe879c96dad899924e26912e19a1dc76078832f84305e8d5342f3445: Status 404 returned error can't find the container with id 5e954479fe879c96dad899924e26912e19a1dc76078832f84305e8d5342f3445 Jul 01 16:59:59.732853 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:59.732814 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" event={"ID":"69739103-a77b-4018-a8f9-699da1459ee9","Type":"ContainerStarted","Data":"27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae"} Jul 01 16:59:59.733289 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:59.732864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" event={"ID":"69739103-a77b-4018-a8f9-699da1459ee9","Type":"ContainerStarted","Data":"5e954479fe879c96dad899924e26912e19a1dc76078832f84305e8d5342f3445"} Jul 01 16:59:59.733289 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:59.732945 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 16:59:59.769459 ip-10-0-140-168 kubenswrapper[2580]: I0701 16:59:59.769417 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" podStartSLOduration=17.769404918 podStartE2EDuration="17.769404918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:59.769355664 +0000 UTC m=+88.938370579" watchObservedRunningTime="2026-07-01 16:59:59.769404918 +0000 UTC m=+88.938419832" Jul 01 17:00:02.731673 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:02.731631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 17:00:02.732110 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:02.732092 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4566659d-c605-480c-8d0b-580e851b62f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 17:00:02.933626 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:02.933587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 17:00:02.935823 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:02.935798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4566659d-c605-480c-8d0b-580e851b62f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssfv4\" (UID: \"4566659d-c605-480c-8d0b-580e851b62f3\") " pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 17:00:02.975397 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:02.975380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssfv4" Jul 01 17:00:03.113039 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:03.113003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssfv4"] Jul 01 17:00:03.116634 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:03.116605 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4566659d_c605_480c_8d0b_580e851b62f3.slice/crio-b22355ae0dbfb7e53ffa155695fa6c71b9bef498408577bc01f985a790dfe99b WatchSource:0}: Error finding container b22355ae0dbfb7e53ffa155695fa6c71b9bef498408577bc01f985a790dfe99b: Status 404 returned error can't find the container with id b22355ae0dbfb7e53ffa155695fa6c71b9bef498408577bc01f985a790dfe99b Jul 01 17:00:03.745431 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:03.745402 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfv4" event={"ID":"4566659d-c605-480c-8d0b-580e851b62f3","Type":"ContainerStarted","Data":"b22355ae0dbfb7e53ffa155695fa6c71b9bef498408577bc01f985a790dfe99b"} Jul 01 17:00:07.759920 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:07.759878 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfv4" event={"ID":"4566659d-c605-480c-8d0b-580e851b62f3","Type":"ContainerStarted","Data":"0c83cbc5bb74974b74208c0f1d3bf0590ad51ab3b0f924a9f9ac79e8117ff04b"} Jul 01 17:00:08.336281 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:08.336256 2580 scope.go:117] "RemoveContainer" containerID="1fbdebe5266815332490ce5c2a31d29a312b7acfae0ff07a6ab0967cb026834c" Jul 01 17:00:08.765515 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:08.765486 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:00:08.765928 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:08.765533 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" event={"ID":"362fbfdc-5a59-4933-b0cf-384af6eba934","Type":"ContainerStarted","Data":"e1aaef12fb549e68d2bcd2f58bb800bd7a1b46e78fc5984755eb57eafbae26d2"} Jul 01 17:00:08.765928 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:08.765891 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 17:00:08.770235 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:08.770214 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" Jul 01 17:00:08.820798 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:08.820753 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-mgmvf" podStartSLOduration=24.57444536 podStartE2EDuration="26.82073725s" totalImagesPullingTime="2.24629189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:08.819479069 +0000 UTC m=+97.988493982" watchObservedRunningTime="2026-07-01 17:00:08.82073725 +0000 UTC m=+97.989752177" Jul 01 17:00:09.770738 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:09.770700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfv4" event={"ID":"4566659d-c605-480c-8d0b-580e851b62f3","Type":"ContainerStarted","Data":"4ddf8c8efbd9c7000634f267c286bc511785bef4b55df1ca81c0b3cf38c56e7d"} Jul 01 17:00:10.591422 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:10.591394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 17:00:10.593895 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:10.593872 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3a7cc17-7e6f-4a93-ba6b-f01760f73a48-metrics-tls\") pod \"dns-default-8q8qz\" (UID: \"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48\") " pod="openshift-dns/dns-default-8q8qz" Jul 01 17:00:10.894395 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:10.894321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wm6rt\"" Jul 01 17:00:10.901540 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:10.901515 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8q8qz" Jul 01 17:00:11.524818 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:11.524794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8q8qz"] Jul 01 17:00:11.527655 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:11.527635 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a7cc17_7e6f_4a93_ba6b_f01760f73a48.slice/crio-6f41dda9bf9f31277c46fd5bf6ee1daf81c92e1332cedcdf11bfffa6c3045acf WatchSource:0}: Error finding container 6f41dda9bf9f31277c46fd5bf6ee1daf81c92e1332cedcdf11bfffa6c3045acf: Status 404 returned error can't find the container with id 6f41dda9bf9f31277c46fd5bf6ee1daf81c92e1332cedcdf11bfffa6c3045acf Jul 01 17:00:11.782473 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:11.782435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8q8qz" event={"ID":"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48","Type":"ContainerStarted","Data":"6f41dda9bf9f31277c46fd5bf6ee1daf81c92e1332cedcdf11bfffa6c3045acf"} Jul 01 17:00:11.784169 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:11.784148 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssfv4" event={"ID":"4566659d-c605-480c-8d0b-580e851b62f3","Type":"ContainerStarted","Data":"269bd32953661e20f0e56326fe12ebf4d5a9421c316015582180186a4b019cf2"} Jul 01 17:00:11.808151 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:11.808105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ssfv4" podStartSLOduration=17.586129952 podStartE2EDuration="25.808092699s" totalImagesPullingTime="8.221962747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:59:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:11.806749461 +0000 UTC m=+100.975764375" watchObservedRunningTime="2026-07-01 17:00:11.808092699 +0000 UTC m=+100.977107614" Jul 01 17:00:13.121892 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.121862 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68"] Jul 01 17:00:13.125046 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.125028 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.137281 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.137254 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 17:00:13.137281 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.137271 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 17:00:13.137420 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.137369 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-774d5bc7bd-87mt4"] Jul 01 17:00:13.137420 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.137409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 01 17:00:13.137566 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.137533 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rr7xx\"" Jul 01 17:00:13.137652 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.137635 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 17:00:13.142039 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.142009 2580 patch_prober.go:28] interesting pod/image-registry-774d5bc7bd-87mt4 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 17:00:13.142145 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.142059 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" podUID="69739103-a77b-4018-a8f9-699da1459ee9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:00:13.157606 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.157583 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68"] Jul 01 17:00:13.205747 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.205716 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf"] Jul 01 17:00:13.208984 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.208940 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.210971 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.210928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/868b2054-3e82-4276-bdb5-e64743ba5f4a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68\" (UID: \"868b2054-3e82-4276-bdb5-e64743ba5f4a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.211081 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.211025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qshxx\" (UniqueName: \"kubernetes.io/projected/868b2054-3e82-4276-bdb5-e64743ba5f4a-kube-api-access-qshxx\") pod \"managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68\" (UID: \"868b2054-3e82-4276-bdb5-e64743ba5f4a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.212491 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.212470 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mtgcn\"" Jul 01 17:00:13.212847 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.212820 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 01 17:00:13.219628 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.219599 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 01 17:00:13.238710 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.238688 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf"] Jul 01 17:00:13.312197 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.312168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-qq4bf\" (UID: \"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.312361 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.312250 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/868b2054-3e82-4276-bdb5-e64743ba5f4a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68\" (UID: \"868b2054-3e82-4276-bdb5-e64743ba5f4a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.312361 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.312301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-qq4bf\" (UID: \"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.312361 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.312333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qshxx\" (UniqueName: \"kubernetes.io/projected/868b2054-3e82-4276-bdb5-e64743ba5f4a-kube-api-access-qshxx\") pod \"managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68\" (UID: \"868b2054-3e82-4276-bdb5-e64743ba5f4a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.314806 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.314783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/868b2054-3e82-4276-bdb5-e64743ba5f4a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68\" (UID: \"868b2054-3e82-4276-bdb5-e64743ba5f4a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.354568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.354532 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qshxx\" (UniqueName: \"kubernetes.io/projected/868b2054-3e82-4276-bdb5-e64743ba5f4a-kube-api-access-qshxx\") pod \"managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68\" (UID: \"868b2054-3e82-4276-bdb5-e64743ba5f4a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.389541 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.389518 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-868bcc99fc-n78b7"] Jul 01 17:00:13.393753 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.393728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.412624 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.412605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-qq4bf\" (UID: \"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.412710 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.412678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-qq4bf\" (UID: \"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.413164 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.413140 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-qq4bf\" (UID: \"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.414842 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.414823 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-qq4bf\" (UID: \"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.436936 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.436916 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868bcc99fc-n78b7"] Jul 01 17:00:13.442438 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.442413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" Jul 01 17:00:13.513929 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.513903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-trusted-ca\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514067 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.513944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6pf8\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-kube-api-access-d6pf8\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514133 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.514082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-registry-tls\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514133 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.514120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-image-registry-private-configuration\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514229 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.514171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-registry-certificates\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514229 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.514205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-ca-trust-extracted\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514328 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.514234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-installation-pull-secrets\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.514328 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.514280 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-bound-sa-token\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.519409 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.519389 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" Jul 01 17:00:13.615101 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-trusted-ca\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615101 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6pf8\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-kube-api-access-d6pf8\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-registry-tls\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-image-registry-private-configuration\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-registry-certificates\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-ca-trust-extracted\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-installation-pull-secrets\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.615304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-bound-sa-token\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.616032 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.615940 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-ca-trust-extracted\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.616164 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.616131 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-registry-certificates\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.616344 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.616305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-trusted-ca\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.618158 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.618114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-registry-tls\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.619500 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.618284 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-image-registry-private-configuration\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.619500 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.618799 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-installation-pull-secrets\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.620532 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.620479 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68"] Jul 01 17:00:13.623318 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:13.623297 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod868b2054_3e82_4276_bdb5_e64743ba5f4a.slice/crio-33c16fad4acd82a9c213d83fbed1ca9ae1f19daa7151b3476423becf9722d06a WatchSource:0}: Error finding container 33c16fad4acd82a9c213d83fbed1ca9ae1f19daa7151b3476423becf9722d06a: Status 404 returned error can't find the container with id 33c16fad4acd82a9c213d83fbed1ca9ae1f19daa7151b3476423becf9722d06a Jul 01 17:00:13.652763 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.652704 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-bound-sa-token\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.652856 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.652838 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6pf8\" (UniqueName: \"kubernetes.io/projected/8d3fddc1-4a19-4605-b4af-481ad7a5ef69-kube-api-access-d6pf8\") pod \"image-registry-868bcc99fc-n78b7\" (UID: \"8d3fddc1-4a19-4605-b4af-481ad7a5ef69\") " pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.671087 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.671061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dmnzt" Jul 01 17:00:13.712305 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.712286 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:13.714362 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.714341 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf"] Jul 01 17:00:13.721018 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:13.720997 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2d57fd_7a5b_4e1a_97f4_56edf0d181f2.slice/crio-ec50305593153217026913a65f3d5d75993fa1c0562f4dfe9c95e57268aeb978 WatchSource:0}: Error finding container ec50305593153217026913a65f3d5d75993fa1c0562f4dfe9c95e57268aeb978: Status 404 returned error can't find the container with id ec50305593153217026913a65f3d5d75993fa1c0562f4dfe9c95e57268aeb978 Jul 01 17:00:13.793704 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.793661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8q8qz" event={"ID":"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48","Type":"ContainerStarted","Data":"e5fc306d3d21c47347aced894c40ae6f6186c48e776aa52c8dfedb8cc9934900"} Jul 01 17:00:13.793835 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.793813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8q8qz" event={"ID":"e3a7cc17-7e6f-4a93-ba6b-f01760f73a48","Type":"ContainerStarted","Data":"d763025a66d9c4bb1ccb13d5b2daea93e6af8a7ca2bd85fe242719d8989936db"} Jul 01 17:00:13.793908 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.793864 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8q8qz" Jul 01 17:00:13.794856 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.794834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" event={"ID":"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2","Type":"ContainerStarted","Data":"ec50305593153217026913a65f3d5d75993fa1c0562f4dfe9c95e57268aeb978"} Jul 01 17:00:13.795851 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.795832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" event={"ID":"868b2054-3e82-4276-bdb5-e64743ba5f4a","Type":"ContainerStarted","Data":"33c16fad4acd82a9c213d83fbed1ca9ae1f19daa7151b3476423becf9722d06a"} Jul 01 17:00:13.875132 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.875081 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8q8qz" podStartSLOduration=66.064110251 podStartE2EDuration="1m7.875066057s" totalImagesPullingTime="1.810955806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:13.871911966 +0000 UTC m=+103.040926880" watchObservedRunningTime="2026-07-01 17:00:13.875066057 +0000 UTC m=+103.044080970" Jul 01 17:00:13.924552 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:13.924483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868bcc99fc-n78b7"] Jul 01 17:00:13.927359 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:13.927337 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3fddc1_4a19_4605_b4af_481ad7a5ef69.slice/crio-ba016b4b29cbc23ba7e710e7c2533ca838267b71aa13ca17e7aa684e937add35 WatchSource:0}: Error finding container ba016b4b29cbc23ba7e710e7c2533ca838267b71aa13ca17e7aa684e937add35: Status 404 returned error can't find the container with id ba016b4b29cbc23ba7e710e7c2533ca838267b71aa13ca17e7aa684e937add35 Jul 01 17:00:14.802377 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:14.802336 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" event={"ID":"8d3fddc1-4a19-4605-b4af-481ad7a5ef69","Type":"ContainerStarted","Data":"4eec49f1fe80c44ab450bc0b6db8ab075ec47e751d7d059e3d64e7883faaf2a7"} Jul 01 17:00:14.802822 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:14.802399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" event={"ID":"8d3fddc1-4a19-4605-b4af-481ad7a5ef69","Type":"ContainerStarted","Data":"ba016b4b29cbc23ba7e710e7c2533ca838267b71aa13ca17e7aa684e937add35"} Jul 01 17:00:14.802822 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:14.802513 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:14.867325 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:14.867273 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" podStartSLOduration=1.867254044 podStartE2EDuration="1.867254044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:14.865868417 +0000 UTC m=+104.034883370" watchObservedRunningTime="2026-07-01 17:00:14.867254044 +0000 UTC m=+104.036268958" Jul 01 17:00:15.807744 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:15.807705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" event={"ID":"6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2","Type":"ContainerStarted","Data":"e691c7f4361d4c571c3d6b6e4ba4b942d485a199c5fc40bf2d46e930d4a04e69"} Jul 01 17:00:15.846543 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:15.845551 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-qq4bf" podStartSLOduration=1.35051866 podStartE2EDuration="2.845529725s" totalImagesPullingTime="1.495011065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:15.844532565 +0000 UTC m=+105.013547480" watchObservedRunningTime="2026-07-01 17:00:15.845529725 +0000 UTC m=+105.014544641" Jul 01 17:00:15.971767 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:15.971731 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-9s8mk"] Jul 01 17:00:15.996436 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:15.996404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.005755 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.005367 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 17:00:16.005755 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.005551 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 17:00:16.005755 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.005609 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 17:00:16.007187 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.007163 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 17:00:16.007387 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.007245 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 17:00:16.007387 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.007321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cf8lt\"" Jul 01 17:00:16.010586 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.010565 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-9s8mk"] Jul 01 17:00:16.136888 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.136802 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6phb5\" (UniqueName: \"kubernetes.io/projected/2c8fc67a-98cb-429a-92da-2c4237b0b471-kube-api-access-6phb5\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.136888 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.136858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.136888 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.136880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8fc67a-98cb-429a-92da-2c4237b0b471-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.137151 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.137020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.238133 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.238103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6phb5\" (UniqueName: \"kubernetes.io/projected/2c8fc67a-98cb-429a-92da-2c4237b0b471-kube-api-access-6phb5\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.238367 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.238156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.238367 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:00:16.238273 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 01 17:00:16.238367 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:00:16.238334 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls podName:2c8fc67a-98cb-429a-92da-2c4237b0b471 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:16.738315338 +0000 UTC m=+105.907330230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-9s8mk" (UID: "2c8fc67a-98cb-429a-92da-2c4237b0b471") : secret "prometheus-operator-tls" not found Jul 01 17:00:16.238367 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.238352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8fc67a-98cb-429a-92da-2c4237b0b471-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.238630 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.238441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.239273 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.239229 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c8fc67a-98cb-429a-92da-2c4237b0b471-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.241006 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.240981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.264558 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.264535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6phb5\" (UniqueName: \"kubernetes.io/projected/2c8fc67a-98cb-429a-92da-2c4237b0b471-kube-api-access-6phb5\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.741794 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:16.741761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:16.741979 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:00:16.741872 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 01 17:00:16.741979 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:00:16.741931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls podName:2c8fc67a-98cb-429a-92da-2c4237b0b471 nodeName:}" failed. No retries permitted until 2026-07-01 17:00:17.741913192 +0000 UTC m=+106.910928086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-9s8mk" (UID: "2c8fc67a-98cb-429a-92da-2c4237b0b471") : secret "prometheus-operator-tls" not found Jul 01 17:00:17.749149 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:17.749113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:17.751599 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:17.751565 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c8fc67a-98cb-429a-92da-2c4237b0b471-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-9s8mk\" (UID: \"2c8fc67a-98cb-429a-92da-2c4237b0b471\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:17.809571 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:17.809543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" Jul 01 17:00:17.952662 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:17.950303 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-9s8mk"] Jul 01 17:00:18.101852 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:18.101785 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8fc67a_98cb_429a_92da_2c4237b0b471.slice/crio-dfd3016a0a55d8ec5b8973bf98a6263e309f9d3d4a8b74025ed1a3d0c817f993 WatchSource:0}: Error finding container dfd3016a0a55d8ec5b8973bf98a6263e309f9d3d4a8b74025ed1a3d0c817f993: Status 404 returned error can't find the container with id dfd3016a0a55d8ec5b8973bf98a6263e309f9d3d4a8b74025ed1a3d0c817f993 Jul 01 17:00:18.819064 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:18.819029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" event={"ID":"868b2054-3e82-4276-bdb5-e64743ba5f4a","Type":"ContainerStarted","Data":"411aa1f36268672ac4bb642adb734e1e8e847bd1b26b808f0f0a417c5fd64f03"} Jul 01 17:00:18.820097 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:18.820077 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" event={"ID":"2c8fc67a-98cb-429a-92da-2c4237b0b471","Type":"ContainerStarted","Data":"dfd3016a0a55d8ec5b8973bf98a6263e309f9d3d4a8b74025ed1a3d0c817f993"} Jul 01 17:00:18.839424 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:18.839382 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b5d6dc8d6-zrr68" podStartSLOduration=1.3259762560000001 podStartE2EDuration="5.83936433s" totalImagesPullingTime="4.513388074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:18.8377439 +0000 UTC m=+108.006758815" watchObservedRunningTime="2026-07-01 17:00:18.83936433 +0000 UTC m=+108.008379245" Jul 01 17:00:21.831738 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:21.831698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" event={"ID":"2c8fc67a-98cb-429a-92da-2c4237b0b471","Type":"ContainerStarted","Data":"5d255e6af90212dba6571d210a7496b6bc261e2b3140a438548cd83c9556280a"} Jul 01 17:00:21.832201 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:21.831764 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" event={"ID":"2c8fc67a-98cb-429a-92da-2c4237b0b471","Type":"ContainerStarted","Data":"b9fa66be4a1c1b754b6d915c59254386699edb4084342ec2f119a2ea2ce16f5d"} Jul 01 17:00:21.873154 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:21.873108 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-9s8mk" podStartSLOduration=3.33973568 podStartE2EDuration="6.873091992s" totalImagesPullingTime="3.533356312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:21.872647085 +0000 UTC m=+111.041661992" watchObservedRunningTime="2026-07-01 17:00:21.873091992 +0000 UTC m=+111.042106939" Jul 01 17:00:23.142341 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:23.142308 2580 patch_prober.go:28] interesting pod/image-registry-774d5bc7bd-87mt4 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 17:00:23.142738 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:23.142359 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" podUID="69739103-a77b-4018-a8f9-699da1459ee9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:00:23.804168 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:23.804137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8q8qz" Jul 01 17:00:24.849736 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.849700 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kgmwl"] Jul 01 17:00:24.852076 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.852055 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:24.856934 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.856914 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 17:00:24.858296 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.858267 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l7mm4\"" Jul 01 17:00:24.861859 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.861839 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 17:00:24.863578 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.863562 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 17:00:24.930174 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.930147 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml"] Jul 01 17:00:24.932254 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.932239 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:24.939000 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.938953 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 01 17:00:24.940108 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.940089 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 01 17:00:24.940207 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.940193 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b85qm\"" Jul 01 17:00:24.958688 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:24.958665 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml"] Jul 01 17:00:25.001586 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001726 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-tls\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001726 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-sys\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001726 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-textfile\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001726 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-accelerators-collector-config\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001726 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-metrics-client-ca\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001892 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-wtmp\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001892 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-root\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.001892 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.001869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkgpq\" (UniqueName: \"kubernetes.io/projected/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-kube-api-access-hkgpq\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.102883 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.102790 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkgpq\" (UniqueName: \"kubernetes.io/projected/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-kube-api-access-hkgpq\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.102883 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.102839 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5mq5\" (UniqueName: \"kubernetes.io/projected/3da6bdfd-df66-465c-ae5d-af1146636a5b-kube-api-access-h5mq5\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.102883 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.102869 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.102895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-tls\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.102938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-sys\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.102992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-textfile\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3da6bdfd-df66-465c-ae5d-af1146636a5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103038 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-sys\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-accelerators-collector-config\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-metrics-client-ca\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-wtmp\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103186 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-root\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103501 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3da6bdfd-df66-465c-ae5d-af1146636a5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.103501 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da6bdfd-df66-465c-ae5d-af1146636a5b-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.103501 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103288 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-wtmp\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103501 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103292 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-root\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103501 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103304 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-textfile\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103726 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103708 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-accelerators-collector-config\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.103991 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.103950 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-metrics-client-ca\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.105412 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.105394 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-tls\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.105508 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.105432 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.111264 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.111241 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkgpq\" (UniqueName: \"kubernetes.io/projected/a9493dd9-f5d6-4afe-b3d9-88c1a8504905-kube-api-access-hkgpq\") pod \"node-exporter-kgmwl\" (UID: \"a9493dd9-f5d6-4afe-b3d9-88c1a8504905\") " pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.161386 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.161361 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kgmwl" Jul 01 17:00:25.170801 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:25.170776 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9493dd9_f5d6_4afe_b3d9_88c1a8504905.slice/crio-9d6da9decbbabeb769791cb7b0166843694c5d817d707ae1e82d1011ab0ac50b WatchSource:0}: Error finding container 9d6da9decbbabeb769791cb7b0166843694c5d817d707ae1e82d1011ab0ac50b: Status 404 returned error can't find the container with id 9d6da9decbbabeb769791cb7b0166843694c5d817d707ae1e82d1011ab0ac50b Jul 01 17:00:25.203877 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.203854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3da6bdfd-df66-465c-ae5d-af1146636a5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.204005 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.203906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3da6bdfd-df66-465c-ae5d-af1146636a5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.204059 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.204001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da6bdfd-df66-465c-ae5d-af1146636a5b-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.204104 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.204063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5mq5\" (UniqueName: \"kubernetes.io/projected/3da6bdfd-df66-465c-ae5d-af1146636a5b-kube-api-access-h5mq5\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.204720 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.204699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da6bdfd-df66-465c-ae5d-af1146636a5b-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.206363 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.206345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3da6bdfd-df66-465c-ae5d-af1146636a5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.206454 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.206426 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3da6bdfd-df66-465c-ae5d-af1146636a5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.225780 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.225756 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5mq5\" (UniqueName: \"kubernetes.io/projected/3da6bdfd-df66-465c-ae5d-af1146636a5b-kube-api-access-h5mq5\") pod \"openshift-state-metrics-57f99c8b49-2kqml\" (UID: \"3da6bdfd-df66-465c-ae5d-af1146636a5b\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.240881 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.240860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" Jul 01 17:00:25.390574 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.390502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml"] Jul 01 17:00:25.394989 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:25.394934 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da6bdfd_df66_465c_ae5d_af1146636a5b.slice/crio-0bb0f2ddd7479fe5608c2dc2c4233fed28ec53f98bf25770f8dc0a3ba4496571 WatchSource:0}: Error finding container 0bb0f2ddd7479fe5608c2dc2c4233fed28ec53f98bf25770f8dc0a3ba4496571: Status 404 returned error can't find the container with id 0bb0f2ddd7479fe5608c2dc2c4233fed28ec53f98bf25770f8dc0a3ba4496571 Jul 01 17:00:25.847474 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.847436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" event={"ID":"3da6bdfd-df66-465c-ae5d-af1146636a5b","Type":"ContainerStarted","Data":"8243a235237ce59b494e245dca655544918ba05966c761061fc065f3a00a093c"} Jul 01 17:00:25.847647 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.847511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" event={"ID":"3da6bdfd-df66-465c-ae5d-af1146636a5b","Type":"ContainerStarted","Data":"a8f203d52e5d0336f940f77f5b19b7ecdd85de6339622e0d6412ebfef4d77744"} Jul 01 17:00:25.847647 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.847542 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" event={"ID":"3da6bdfd-df66-465c-ae5d-af1146636a5b","Type":"ContainerStarted","Data":"0bb0f2ddd7479fe5608c2dc2c4233fed28ec53f98bf25770f8dc0a3ba4496571"} Jul 01 17:00:25.848521 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:25.848499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgmwl" event={"ID":"a9493dd9-f5d6-4afe-b3d9-88c1a8504905","Type":"ContainerStarted","Data":"9d6da9decbbabeb769791cb7b0166843694c5d817d707ae1e82d1011ab0ac50b"} Jul 01 17:00:26.855364 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:26.855328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" event={"ID":"3da6bdfd-df66-465c-ae5d-af1146636a5b","Type":"ContainerStarted","Data":"11b75bda934d16b6ebc9eca7d5ee21c0e604cb0b98630e8360b94fd0308bd884"} Jul 01 17:00:26.883044 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:26.882940 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-2kqml" podStartSLOduration=1.954078413 podStartE2EDuration="2.882916958s" totalImagesPullingTime="928.838545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:26.881162511 +0000 UTC m=+116.050177429" watchObservedRunningTime="2026-07-01 17:00:26.882916958 +0000 UTC m=+116.051931874" Jul 01 17:00:27.860763 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:27.860725 2580 generic.go:356] "Generic (PLEG): container finished" podID="a9493dd9-f5d6-4afe-b3d9-88c1a8504905" containerID="57d095430cfc0d63683b8cee01736fe3c39f19ce353e778124f0b7c3a55869f7" exitCode=0 Jul 01 17:00:27.861257 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:27.860834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgmwl" event={"ID":"a9493dd9-f5d6-4afe-b3d9-88c1a8504905","Type":"ContainerDied","Data":"57d095430cfc0d63683b8cee01736fe3c39f19ce353e778124f0b7c3a55869f7"} Jul 01 17:00:28.867124 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:28.867082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgmwl" event={"ID":"a9493dd9-f5d6-4afe-b3d9-88c1a8504905","Type":"ContainerStarted","Data":"01b0cc6ca58315ae5137afe5184e459dfd0c88994a98b40c3f7e94f07c1323a9"} Jul 01 17:00:28.867493 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:28.867157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kgmwl" event={"ID":"a9493dd9-f5d6-4afe-b3d9-88c1a8504905","Type":"ContainerStarted","Data":"0b8f1c96f1d26760d688e64664b16968a88ea066dc5d0bddf14b9bf514c2d394"} Jul 01 17:00:28.890533 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:28.890490 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kgmwl" podStartSLOduration=2.963782991 podStartE2EDuration="4.890476263s" totalImagesPullingTime="1.855528939s" totalInitContainerRuntime="71.164333ms" isStatefulPod=true podCreationTimestamp="2026-07-01 17:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:28.889118925 +0000 UTC m=+118.058133831" watchObservedRunningTime="2026-07-01 17:00:28.890476263 +0000 UTC m=+118.059491177" Jul 01 17:00:30.148895 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.148859 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-56f867fbc8-rhp4n"] Jul 01 17:00:30.151378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.151361 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.154445 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.154423 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 01 17:00:30.154445 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.154441 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-45x6f\"" Jul 01 17:00:30.154923 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.154894 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 01 17:00:30.155028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.155015 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 01 17:00:30.155455 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.155440 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 01 17:00:30.155527 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.155513 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 01 17:00:30.161384 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.161366 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 01 17:00:30.169274 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.169250 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56f867fbc8-rhp4n"] Jul 01 17:00:30.251977 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.251925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-telemeter-client-tls\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.251977 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.251982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.252179 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.252032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-serving-certs-ca-bundle\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.252179 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.252071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485wd\" (UniqueName: \"kubernetes.io/projected/caa7ec7b-b150-402a-b3a5-ab4329132793-kube-api-access-485wd\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.252179 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.252121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-secret-telemeter-client\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.252284 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.252196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-federate-client-tls\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.252284 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.252223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-metrics-client-ca\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.252284 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.252253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353440 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-metrics-client-ca\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353575 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353628 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-telemeter-client-tls\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353663 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353701 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-serving-certs-ca-bundle\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353701 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-485wd\" (UniqueName: \"kubernetes.io/projected/caa7ec7b-b150-402a-b3a5-ab4329132793-kube-api-access-485wd\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353780 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-secret-telemeter-client\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.353867 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.353846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-federate-client-tls\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.354289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.354257 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-metrics-client-ca\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.354451 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.354427 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.354693 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.354566 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caa7ec7b-b150-402a-b3a5-ab4329132793-serving-certs-ca-bundle\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.356288 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.356258 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.356288 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.356279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-federate-client-tls\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.356441 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.356338 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-secret-telemeter-client\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.356441 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.356364 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/caa7ec7b-b150-402a-b3a5-ab4329132793-telemeter-client-tls\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.366408 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.366383 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-485wd\" (UniqueName: \"kubernetes.io/projected/caa7ec7b-b150-402a-b3a5-ab4329132793-kube-api-access-485wd\") pod \"telemeter-client-56f867fbc8-rhp4n\" (UID: \"caa7ec7b-b150-402a-b3a5-ab4329132793\") " pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.460801 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.460718 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" Jul 01 17:00:30.611208 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.611176 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56f867fbc8-rhp4n"] Jul 01 17:00:30.614483 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:30.614454 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa7ec7b_b150_402a_b3a5_ab4329132793.slice/crio-5415caf4a967836267b1a0b90ce089aaaa7cbec4e19df8a6a73f4fa2afb644ee WatchSource:0}: Error finding container 5415caf4a967836267b1a0b90ce089aaaa7cbec4e19df8a6a73f4fa2afb644ee: Status 404 returned error can't find the container with id 5415caf4a967836267b1a0b90ce089aaaa7cbec4e19df8a6a73f4fa2afb644ee Jul 01 17:00:30.875393 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.875348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" event={"ID":"caa7ec7b-b150-402a-b3a5-ab4329132793","Type":"ContainerStarted","Data":"5415caf4a967836267b1a0b90ce089aaaa7cbec4e19df8a6a73f4fa2afb644ee"} Jul 01 17:00:30.985542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.985508 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:00:30.989442 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.988774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:30.995577 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.995545 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 17:00:30.995815 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.995619 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 17:00:30.995815 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.995671 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 17:00:30.995815 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.995686 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 17:00:30.996039 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.996023 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8dnov1qv2o8qc\"" Jul 01 17:00:30.996763 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.996739 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 17:00:30.996890 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.996744 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 17:00:30.997287 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.997267 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 17:00:30.997510 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.997495 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 17:00:30.999931 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.999912 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b87wc\"" Jul 01 17:00:31.000044 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:30.999913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 17:00:31.000110 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.000091 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 17:00:31.000258 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.000241 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 17:00:31.000433 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.000422 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 17:00:31.000575 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.000557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 17:00:31.003728 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.003712 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 17:00:31.010773 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.010754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 17:00:31.015526 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.015498 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:00:31.160028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.159884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160002 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wln84\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-kube-api-access-wln84\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-config-out\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-config\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.160568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160487 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.161327 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.160513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-web-config\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.260871 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.260842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.260871 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.260875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261092 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.260893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wln84\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-kube-api-access-wln84\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261092 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261092 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261092 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-config-out\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261297 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261297 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261297 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261297 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261297 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-config\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.261593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-web-config\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.262107 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.262107 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.261923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.263552 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.263530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-config-out\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.263879 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.263864 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 17:00:31.263879 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.263873 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 17:00:31.264152 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.264135 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 17:00:31.264226 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.264170 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 17:00:31.264293 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.264277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 17:00:31.264383 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.264366 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8dnov1qv2o8qc\"" Jul 01 17:00:31.264481 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.264466 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 17:00:31.264580 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.264565 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 17:00:31.267144 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.267128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 17:00:31.267249 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.267235 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 17:00:31.267556 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.267504 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 17:00:31.267556 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.267534 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 17:00:31.267700 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.267537 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 17:00:31.267700 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.267505 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 17:00:31.271551 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.271530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.271691 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.271669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.271859 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.271841 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 17:00:31.274313 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.274287 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.274419 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.274395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.274492 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.274433 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.275289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.274802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wln84\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-kube-api-access-wln84\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.275289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.274807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.275289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.274944 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 17:00:31.275289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.275077 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.275289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.275128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.275289 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.275220 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.277152 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.277050 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.277667 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.277626 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.277744 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.277683 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-web-config\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.278285 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.278261 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.278510 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.278485 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-config\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.282318 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.282273 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.284466 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.284378 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.304214 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.304179 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b87wc\"" Jul 01 17:00:31.312736 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.312703 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:00:31.493257 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.493221 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:00:31.496319 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:31.496295 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5899b3fa_e499_4570_80ee_92a1c17f3d33.slice/crio-b36f6a64fdb14ae6a9ab702ab3f43eaf7e2537b1ec382feb29a77cb42c5c0077 WatchSource:0}: Error finding container b36f6a64fdb14ae6a9ab702ab3f43eaf7e2537b1ec382feb29a77cb42c5c0077: Status 404 returned error can't find the container with id b36f6a64fdb14ae6a9ab702ab3f43eaf7e2537b1ec382feb29a77cb42c5c0077 Jul 01 17:00:31.880492 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:31.880455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"b36f6a64fdb14ae6a9ab702ab3f43eaf7e2537b1ec382feb29a77cb42c5c0077"} Jul 01 17:00:33.142398 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.142369 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 17:00:33.890752 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.890705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" event={"ID":"caa7ec7b-b150-402a-b3a5-ab4329132793","Type":"ContainerStarted","Data":"3b05cfb4fc16bf88f30bde72bf112b8d1e27b66d29e319309ea2ec4c92ef0dc6"} Jul 01 17:00:33.890924 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.890782 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" event={"ID":"caa7ec7b-b150-402a-b3a5-ab4329132793","Type":"ContainerStarted","Data":"d8fb3372a9b6ce76e402802dfe578a74976d22c79756dd138a99a14773a861bf"} Jul 01 17:00:33.890924 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.890809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" event={"ID":"caa7ec7b-b150-402a-b3a5-ab4329132793","Type":"ContainerStarted","Data":"7dd21e5b2af1f5fba0678144cd333167f3ea158ef620c43fddc20f63b4e80b44"} Jul 01 17:00:33.892338 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.892304 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" exitCode=0 Jul 01 17:00:33.892438 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.892350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} Jul 01 17:00:33.916793 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:33.916749 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-56f867fbc8-rhp4n" podStartSLOduration=1.699245546 podStartE2EDuration="3.916733153s" totalImagesPullingTime="2.217487607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:33.914927196 +0000 UTC m=+123.083942111" watchObservedRunningTime="2026-07-01 17:00:33.916733153 +0000 UTC m=+123.085748067" Jul 01 17:00:35.812697 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:35.812665 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-868bcc99fc-n78b7" Jul 01 17:00:37.911280 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:37.911247 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} Jul 01 17:00:38.158463 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.158374 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" podUID="69739103-a77b-4018-a8f9-699da1459ee9" containerName="registry" containerID="cri-o://27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae" gracePeriod=30 Jul 01 17:00:38.388799 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.388776 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 17:00:38.423117 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423042 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-trusted-ca\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423117 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423082 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423117 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423108 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2kb6\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-kube-api-access-k2kb6\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423133 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-image-registry-private-configuration\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423161 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-installation-pull-secrets\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423222 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-bound-sa-token\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423266 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69739103-a77b-4018-a8f9-699da1459ee9-ca-trust-extracted\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423297 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-registry-certificates\") pod \"69739103-a77b-4018-a8f9-699da1459ee9\" (UID: \"69739103-a77b-4018-a8f9-699da1459ee9\") " Jul 01 17:00:38.423830 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423516 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:38.423992 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.423951 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:38.426261 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.426163 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:38.426362 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.426309 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-kube-api-access-k2kb6" (OuterVolumeSpecName: "kube-api-access-k2kb6") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "kube-api-access-k2kb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:38.426560 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.426532 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:00:38.426662 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.426590 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:00:38.427159 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.427134 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:38.433507 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.433478 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69739103-a77b-4018-a8f9-699da1459ee9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "69739103-a77b-4018-a8f9-699da1459ee9" (UID: "69739103-a77b-4018-a8f9-699da1459ee9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:00:38.524945 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.524911 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69739103-a77b-4018-a8f9-699da1459ee9-ca-trust-extracted\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.524945 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.524941 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-registry-certificates\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.524945 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.524950 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69739103-a77b-4018-a8f9-699da1459ee9-trusted-ca\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.525178 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.524981 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-registry-tls\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.525178 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.524989 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2kb6\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-kube-api-access-k2kb6\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.525178 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.524998 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-image-registry-private-configuration\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.525178 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.525007 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69739103-a77b-4018-a8f9-699da1459ee9-installation-pull-secrets\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.525178 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.525015 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69739103-a77b-4018-a8f9-699da1459ee9-bound-sa-token\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:00:38.918579 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.918546 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} Jul 01 17:00:38.919913 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.919890 2580 generic.go:356] "Generic (PLEG): container finished" podID="69739103-a77b-4018-a8f9-699da1459ee9" containerID="27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae" exitCode=0 Jul 01 17:00:38.920028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.919939 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" Jul 01 17:00:38.920028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.919974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" event={"ID":"69739103-a77b-4018-a8f9-699da1459ee9","Type":"ContainerDied","Data":"27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae"} Jul 01 17:00:38.920028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.920011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-774d5bc7bd-87mt4" event={"ID":"69739103-a77b-4018-a8f9-699da1459ee9","Type":"ContainerDied","Data":"5e954479fe879c96dad899924e26912e19a1dc76078832f84305e8d5342f3445"} Jul 01 17:00:38.920139 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.920033 2580 scope.go:117] "RemoveContainer" containerID="27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae" Jul 01 17:00:38.929277 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.929258 2580 scope.go:117] "RemoveContainer" containerID="27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae" Jul 01 17:00:38.929600 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:00:38.929572 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae\": container with ID starting with 27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae not found: ID does not exist" containerID="27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae" Jul 01 17:00:38.929921 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.929609 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae"} err="failed to get container status \"27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae\": rpc error: code = NotFound desc = could not find container \"27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae\": container with ID starting with 27a4691db2604bbc68fc59bf8692687f282a4629fd7ae5d1b27af367064608ae not found: ID does not exist" Jul 01 17:00:38.946369 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.946344 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-774d5bc7bd-87mt4"] Jul 01 17:00:38.950163 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:38.950134 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-774d5bc7bd-87mt4"] Jul 01 17:00:39.342994 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:39.342940 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69739103-a77b-4018-a8f9-699da1459ee9" path="/var/lib/kubelet/pods/69739103-a77b-4018-a8f9-699da1459ee9/volumes" Jul 01 17:00:41.144013 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.143976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 17:00:41.146639 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.146601 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13ee94a-dccb-417d-9067-ba4b8de586ef-metrics-certs\") pod \"network-metrics-daemon-w4kj7\" (UID: \"f13ee94a-dccb-417d-9067-ba4b8de586ef\") " pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 17:00:41.355542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.355515 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6c472\"" Jul 01 17:00:41.362872 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.362839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4kj7" Jul 01 17:00:41.726132 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.726107 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4kj7"] Jul 01 17:00:41.740181 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:00:41.740153 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13ee94a_dccb_417d_9067_ba4b8de586ef.slice/crio-5409ba51120ef55df27a2ae199ab2e83fb4d94c4d976ddb9a7ec41a0d11c6c1d WatchSource:0}: Error finding container 5409ba51120ef55df27a2ae199ab2e83fb4d94c4d976ddb9a7ec41a0d11c6c1d: Status 404 returned error can't find the container with id 5409ba51120ef55df27a2ae199ab2e83fb4d94c4d976ddb9a7ec41a0d11c6c1d Jul 01 17:00:41.932808 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.932772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4kj7" event={"ID":"f13ee94a-dccb-417d-9067-ba4b8de586ef","Type":"ContainerStarted","Data":"5409ba51120ef55df27a2ae199ab2e83fb4d94c4d976ddb9a7ec41a0d11c6c1d"} Jul 01 17:00:41.935864 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.935836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} Jul 01 17:00:41.935988 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.935904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} Jul 01 17:00:41.935988 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.935944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} Jul 01 17:00:41.936073 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.936001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerStarted","Data":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} Jul 01 17:00:41.973294 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:41.970104 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.99823494 podStartE2EDuration="11.970086044s" totalImagesPullingTime="8.944796542s" totalInitContainerRuntime="27.054562ms" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:41.968705494 +0000 UTC m=+131.137720423" watchObservedRunningTime="2026-07-01 17:00:41.970086044 +0000 UTC m=+131.139100962" Jul 01 17:00:44.949413 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:44.949378 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4kj7" event={"ID":"f13ee94a-dccb-417d-9067-ba4b8de586ef","Type":"ContainerStarted","Data":"84c825f6f9ee5723e25066d19fda8757e97ca616c4c1c5924c8b564406b7161b"} Jul 01 17:00:44.949782 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:44.949461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4kj7" event={"ID":"f13ee94a-dccb-417d-9067-ba4b8de586ef","Type":"ContainerStarted","Data":"e4af7a1acfe589bc71a2bff060221c74092d8bf499bf3ef6aa6918041288e8b8"} Jul 01 17:00:44.971682 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:44.971635 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w4kj7" podStartSLOduration=130.305722175 podStartE2EDuration="2m12.971620584s" totalImagesPullingTime="2.665898409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:44.970336743 +0000 UTC m=+134.139351658" watchObservedRunningTime="2026-07-01 17:00:44.971620584 +0000 UTC m=+134.140635498" Jul 01 17:00:46.313828 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:00:46.313795 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:11.048617 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:11.048581 2580 generic.go:356] "Generic (PLEG): container finished" podID="e4cde9da-cbdc-4018-8639-7656ff9a614f" containerID="70b4cc3e9205e4c73b22a6e3aa242f1ca84aea121bfa2eee3bfa5c7444a5d912" exitCode=0 Jul 01 17:01:11.049065 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:11.048624 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" event={"ID":"e4cde9da-cbdc-4018-8639-7656ff9a614f","Type":"ContainerDied","Data":"70b4cc3e9205e4c73b22a6e3aa242f1ca84aea121bfa2eee3bfa5c7444a5d912"} Jul 01 17:01:11.049065 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:11.048980 2580 scope.go:117] "RemoveContainer" containerID="70b4cc3e9205e4c73b22a6e3aa242f1ca84aea121bfa2eee3bfa5c7444a5d912" Jul 01 17:01:12.054403 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:12.054369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lxtr5" event={"ID":"e4cde9da-cbdc-4018-8639-7656ff9a614f","Type":"ContainerStarted","Data":"219c6cfbddcabcc79f871b47bee9ef38cd05d0916dadae449310d1d23654e95b"} Jul 01 17:01:31.313433 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:31.313399 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:31.333453 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:31.333432 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:32.141698 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:32.141667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:49.346613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.346580 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:01:49.347156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.346851 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="prometheus" containerID="cri-o://5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" gracePeriod=600 Jul 01 17:01:49.347156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.346886 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="thanos-sidecar" containerID="cri-o://74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" gracePeriod=600 Jul 01 17:01:49.347156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.346887 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy" containerID="cri-o://0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" gracePeriod=600 Jul 01 17:01:49.347156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.346913 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="config-reloader" containerID="cri-o://bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" gracePeriod=600 Jul 01 17:01:49.347156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.346951 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-web" containerID="cri-o://e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" gracePeriod=600 Jul 01 17:01:49.347156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.347048 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" gracePeriod=600 Jul 01 17:01:49.595192 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.595167 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:49.678767 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678684 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-metrics-client-ca\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.678767 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678716 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-grpc-tls\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.678767 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678751 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-thanos-prometheus-http-client-file\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678782 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-serving-certs-ca-bundle\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678815 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678844 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-db\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678869 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-trusted-ca-bundle\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678894 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-0\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678919 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-tls\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.678992 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-kubelet-serving-ca-bundle\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679022 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-1\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679055 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679049 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679083 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-kube-rbac-proxy\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679105 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-config\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679101 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679136 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-tls-assets\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679175 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-web-config\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679184 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679209 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-2\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679240 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wln84\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-kube-api-access-wln84\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679267 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-metrics-client-certs\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679300 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-config-out\") pod \"5899b3fa-e499-4570-80ee-92a1c17f3d33\" (UID: \"5899b3fa-e499-4570-80ee-92a1c17f3d33\") " Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679363 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679434 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679577 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-metrics-client-ca\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679595 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679613 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.679613 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.679629 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.680468 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.680285 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.681091 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.680791 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:01:49.682343 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.682060 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.683542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683365 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.683542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683364 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.683542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683435 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.683542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683448 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-config-out" (OuterVolumeSpecName: "config-out") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:01:49.683542 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683518 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.683871 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683625 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.683871 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683696 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:01:49.683871 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.683699 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:01:49.684213 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.684175 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.684705 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.684678 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-config" (OuterVolumeSpecName: "config") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.684777 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.684698 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-kube-api-access-wln84" (OuterVolumeSpecName: "kube-api-access-wln84") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "kube-api-access-wln84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:01:49.685647 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.685628 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.695363 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.695342 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-web-config" (OuterVolumeSpecName: "web-config") pod "5899b3fa-e499-4570-80ee-92a1c17f3d33" (UID: "5899b3fa-e499-4570-80ee-92a1c17f3d33"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:01:49.779952 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.779922 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.779952 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.779948 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.779984 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-kube-rbac-proxy\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.779998 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-config\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780006 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-tls-assets\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780014 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-web-config\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780021 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780029 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wln84\" (UniqueName: \"kubernetes.io/projected/5899b3fa-e499-4570-80ee-92a1c17f3d33-kube-api-access-wln84\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780037 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-metrics-client-certs\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780045 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-config-out\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780057 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-grpc-tls\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780065 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780074 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780082 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-db\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780091 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5899b3fa-e499-4570-80ee-92a1c17f3d33-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:49.780125 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:49.780099 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5899b3fa-e499-4570-80ee-92a1c17f3d33-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:01:50.194878 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194850 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" exitCode=0 Jul 01 17:01:50.194878 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194873 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" exitCode=0 Jul 01 17:01:50.194878 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194879 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" exitCode=0 Jul 01 17:01:50.194878 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194884 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" exitCode=0 Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194889 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" exitCode=0 Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194894 2580 generic.go:356] "Generic (PLEG): container finished" podID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" exitCode=0 Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194932 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.194932 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195038 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195092 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195103 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.195136 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} Jul 01 17:01:50.195495 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} Jul 01 17:01:50.195495 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.195182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5899b3fa-e499-4570-80ee-92a1c17f3d33","Type":"ContainerDied","Data":"b36f6a64fdb14ae6a9ab702ab3f43eaf7e2537b1ec382feb29a77cb42c5c0077"} Jul 01 17:01:50.203514 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.203493 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.210104 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.210088 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.216772 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.216757 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.223155 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.223133 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.224811 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.224789 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:01:50.230093 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.230078 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.235427 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.235409 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:01:50.240137 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.240122 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.246428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.246414 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.246665 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.246646 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.246711 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.246671 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} err="failed to get container status \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" Jul 01 17:01:50.246711 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.246689 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.246923 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.246905 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.247022 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.246931 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} err="failed to get container status \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" Jul 01 17:01:50.247022 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.246946 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.247231 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.247212 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.247272 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247235 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} err="failed to get container status \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" Jul 01 17:01:50.247272 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247249 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.247456 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.247440 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.247496 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247460 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} err="failed to get container status \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" Jul 01 17:01:50.247496 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247473 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.247713 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.247696 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.247757 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247718 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} err="failed to get container status \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" Jul 01 17:01:50.247757 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247732 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.247911 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.247895 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.247950 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247913 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} err="failed to get container status \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" Jul 01 17:01:50.247950 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.247927 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.248170 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:01:50.248154 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.248227 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248173 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} err="failed to get container status \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" Jul 01 17:01:50.248227 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248185 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.248393 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248373 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} err="failed to get container status \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" Jul 01 17:01:50.248440 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248394 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.248599 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248582 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} err="failed to get container status \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" Jul 01 17:01:50.248641 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248600 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.248795 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248780 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} err="failed to get container status \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" Jul 01 17:01:50.248839 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248795 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.248992 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248977 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} err="failed to get container status \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" Jul 01 17:01:50.248992 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.248991 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.249167 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249151 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} err="failed to get container status \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" Jul 01 17:01:50.249216 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249168 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.249393 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249376 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} err="failed to get container status \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" Jul 01 17:01:50.249436 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249394 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.249584 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249563 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} err="failed to get container status \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" Jul 01 17:01:50.249653 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249586 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.249787 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249771 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} err="failed to get container status \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" Jul 01 17:01:50.249831 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249787 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.249993 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249973 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} err="failed to get container status \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" Jul 01 17:01:50.250040 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.249994 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.250184 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250167 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} err="failed to get container status \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" Jul 01 17:01:50.250248 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250186 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.250394 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250378 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} err="failed to get container status \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" Jul 01 17:01:50.250443 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250394 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.250593 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250576 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} err="failed to get container status \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" Jul 01 17:01:50.250631 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250593 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.250777 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250763 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} err="failed to get container status \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" Jul 01 17:01:50.250817 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250777 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.250944 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250928 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} err="failed to get container status \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" Jul 01 17:01:50.251085 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.250944 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.251212 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251194 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} err="failed to get container status \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" Jul 01 17:01:50.251257 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251212 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.251389 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251371 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} err="failed to get container status \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" Jul 01 17:01:50.251439 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251390 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.251624 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251605 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} err="failed to get container status \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" Jul 01 17:01:50.251664 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251625 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.251818 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251802 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} err="failed to get container status \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" Jul 01 17:01:50.251876 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.251818 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.252067 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252050 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} err="failed to get container status \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" Jul 01 17:01:50.252127 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252067 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.252290 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252272 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} err="failed to get container status \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" Jul 01 17:01:50.252330 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252290 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.252519 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252503 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} err="failed to get container status \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" Jul 01 17:01:50.252596 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252520 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.252751 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252734 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} err="failed to get container status \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" Jul 01 17:01:50.252793 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252751 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.252979 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252950 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} err="failed to get container status \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" Jul 01 17:01:50.253026 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.252979 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.253176 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253160 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} err="failed to get container status \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" Jul 01 17:01:50.253242 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253176 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.253405 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253387 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} err="failed to get container status \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" Jul 01 17:01:50.253463 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253406 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.253633 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253607 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} err="failed to get container status \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" Jul 01 17:01:50.253633 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253625 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.253859 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253843 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} err="failed to get container status \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" Jul 01 17:01:50.253859 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.253859 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.254071 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254053 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} err="failed to get container status \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" Jul 01 17:01:50.254071 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254070 2580 scope.go:117] "RemoveContainer" containerID="f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624" Jul 01 17:01:50.254251 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254233 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624"} err="failed to get container status \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": rpc error: code = NotFound desc = could not find container \"f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624\": container with ID starting with f1a497aabe79bde62fe43c2a66cb9b0410caee2b6b47a90c4ba1b54a85028624 not found: ID does not exist" Jul 01 17:01:50.254304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254252 2580 scope.go:117] "RemoveContainer" containerID="0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc" Jul 01 17:01:50.254440 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254420 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc"} err="failed to get container status \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": rpc error: code = NotFound desc = could not find container \"0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc\": container with ID starting with 0734bf90b8a792a6b0748bfa9cf74f36fee73d448a7b199ce6d60e8481c3d0bc not found: ID does not exist" Jul 01 17:01:50.254440 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254441 2580 scope.go:117] "RemoveContainer" containerID="e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216" Jul 01 17:01:50.254703 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254687 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216"} err="failed to get container status \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": rpc error: code = NotFound desc = could not find container \"e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216\": container with ID starting with e538056f65eb8501ceb4c649f8a9420a582616c9705c6b798d802e8295f70216 not found: ID does not exist" Jul 01 17:01:50.254756 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254703 2580 scope.go:117] "RemoveContainer" containerID="74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb" Jul 01 17:01:50.254929 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254913 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb"} err="failed to get container status \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": rpc error: code = NotFound desc = could not find container \"74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb\": container with ID starting with 74520feb0eabfb90bb88a9773385afddd6303f02c69c141db3e3affe969a8bdb not found: ID does not exist" Jul 01 17:01:50.254992 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.254929 2580 scope.go:117] "RemoveContainer" containerID="bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d" Jul 01 17:01:50.255180 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.255163 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d"} err="failed to get container status \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": rpc error: code = NotFound desc = could not find container \"bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d\": container with ID starting with bf8f750d843c155205117abd5d6decb665ccf8b290561566076409fff70cde3d not found: ID does not exist" Jul 01 17:01:50.255225 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.255180 2580 scope.go:117] "RemoveContainer" containerID="5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875" Jul 01 17:01:50.255421 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.255406 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875"} err="failed to get container status \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": rpc error: code = NotFound desc = could not find container \"5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875\": container with ID starting with 5b3201919cb9258e76c6aed643efd5d434303cc074ba4177a51b527614b70875 not found: ID does not exist" Jul 01 17:01:50.255481 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.255421 2580 scope.go:117] "RemoveContainer" containerID="82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2" Jul 01 17:01:50.255646 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.255631 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2"} err="failed to get container status \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": rpc error: code = NotFound desc = could not find container \"82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2\": container with ID starting with 82438ba7adf2516c79beae627dc734375d6668e48d22448eb305653330a68ab2 not found: ID does not exist" Jul 01 17:01:50.306707 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.306690 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:01:50.306977 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.306947 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="config-reloader" Jul 01 17:01:50.306977 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.306972 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="config-reloader" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.306988 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69739103-a77b-4018-a8f9-699da1459ee9" containerName="registry" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.306995 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="69739103-a77b-4018-a8f9-699da1459ee9" containerName="registry" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307006 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-web" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307011 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-web" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307021 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="prometheus" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307025 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="prometheus" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307035 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="init-config-reloader" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307040 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="init-config-reloader" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307046 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="thanos-sidecar" Jul 01 17:01:50.307048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307051 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="thanos-sidecar" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307056 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-thanos" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307061 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-thanos" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307068 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307073 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307114 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="thanos-sidecar" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307120 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-thanos" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307127 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307133 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="kube-rbac-proxy-web" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307139 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="69739103-a77b-4018-a8f9-699da1459ee9" containerName="registry" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307145 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="prometheus" Jul 01 17:01:50.307354 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.307153 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" containerName="config-reloader" Jul 01 17:01:50.310480 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.310464 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.324281 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.324256 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 17:01:50.325091 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.325075 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 17:01:50.325196 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.325182 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 17:01:50.334185 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334171 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 17:01:50.334364 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334343 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 17:01:50.334473 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334436 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 17:01:50.334473 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334444 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8dnov1qv2o8qc\"" Jul 01 17:01:50.334589 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334468 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 17:01:50.334589 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334468 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 17:01:50.334673 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334503 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 17:01:50.334673 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b87wc\"" Jul 01 17:01:50.334814 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334801 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 17:01:50.334866 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.334831 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 17:01:50.335275 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.335261 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 17:01:50.335329 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.335264 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 17:01:50.336724 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.336706 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 17:01:50.344057 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.344042 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 17:01:50.381757 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.381733 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:01:50.385246 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385329 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385329 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3f0893a6-c147-497e-b662-88d662ef5f3e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385329 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385291 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3f0893a6-c147-497e-b662-88d662ef5f3e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385329 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4hd5\" (UniqueName: \"kubernetes.io/projected/3f0893a6-c147-497e-b662-88d662ef5f3e-kube-api-access-j4hd5\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385582 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385639 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-config\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.385857 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.385834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486353 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486484 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486484 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486484 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486484 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486484 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486484 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-config\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486880 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486857 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.486992 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.486974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487059 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487059 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487046 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487170 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487170 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3f0893a6-c147-497e-b662-88d662ef5f3e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3f0893a6-c147-497e-b662-88d662ef5f3e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4hd5\" (UniqueName: \"kubernetes.io/projected/3f0893a6-c147-497e-b662-88d662ef5f3e-kube-api-access-j4hd5\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.487428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.488459 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.488179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.488556 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.488542 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.490008 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.487046 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.490008 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.489816 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.491402 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.490595 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.491402 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.491001 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-config\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.491566 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.491444 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.491923 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.491742 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.491923 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.491858 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3f0893a6-c147-497e-b662-88d662ef5f3e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.491923 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.491879 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3f0893a6-c147-497e-b662-88d662ef5f3e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.492112 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.491980 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.492112 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.492087 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.492236 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.492214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.492298 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.492282 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.492943 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.492929 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3f0893a6-c147-497e-b662-88d662ef5f3e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.493294 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.493279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3f0893a6-c147-497e-b662-88d662ef5f3e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.496866 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.496846 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4hd5\" (UniqueName: \"kubernetes.io/projected/3f0893a6-c147-497e-b662-88d662ef5f3e-kube-api-access-j4hd5\") pod \"prometheus-k8s-0\" (UID: \"3f0893a6-c147-497e-b662-88d662ef5f3e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.619847 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.619819 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:01:50.768457 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:50.768380 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 17:01:50.771085 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:01:50.771063 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0893a6_c147_497e_b662_88d662ef5f3e.slice/crio-a4a0be36a2e2b8a45d7829f64b8f4c20f3529893ba27edb9b91ba0512721e1bb WatchSource:0}: Error finding container a4a0be36a2e2b8a45d7829f64b8f4c20f3529893ba27edb9b91ba0512721e1bb: Status 404 returned error can't find the container with id a4a0be36a2e2b8a45d7829f64b8f4c20f3529893ba27edb9b91ba0512721e1bb Jul 01 17:01:51.200945 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:51.200871 2580 generic.go:356] "Generic (PLEG): container finished" podID="3f0893a6-c147-497e-b662-88d662ef5f3e" containerID="13e6ae8e4e12957b8755a743b0a9363024c8cd157a02a5d455717e44314c1ca8" exitCode=0 Jul 01 17:01:51.201131 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:51.200984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerDied","Data":"13e6ae8e4e12957b8755a743b0a9363024c8cd157a02a5d455717e44314c1ca8"} Jul 01 17:01:51.201131 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:51.201063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"a4a0be36a2e2b8a45d7829f64b8f4c20f3529893ba27edb9b91ba0512721e1bb"} Jul 01 17:01:51.346978 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:51.344418 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5899b3fa-e499-4570-80ee-92a1c17f3d33" path="/var/lib/kubelet/pods/5899b3fa-e499-4570-80ee-92a1c17f3d33/volumes" Jul 01 17:01:52.208257 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.208228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"62cc814f147130c841364ce5feebfc57cce0a0d73474eecc4ea8d9274d5b5548"} Jul 01 17:01:52.208604 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.208290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"56a6e14994d59ccdea360eb99e5fac7e78a0ab3818f4b701de73ff1ca5e16426"} Jul 01 17:01:52.208604 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.208317 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"700325b051b5ab133bfc859b5b3a22ac5cf3fc5c8d41015be800b1ab1d67c99d"} Jul 01 17:01:52.208604 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.208340 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"5cc15296cef33bb94f0c7112555c19cd77c837d437b6c7e030a2f210bb2aeaa5"} Jul 01 17:01:52.208604 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.208363 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"e6b566b0d1e98a973ca7eeb24c5dea00e1350ac3fd092bae51f2b26db6c5453d"} Jul 01 17:01:52.208604 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.208385 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3f0893a6-c147-497e-b662-88d662ef5f3e","Type":"ContainerStarted","Data":"697e504767bda604afcede2a72bbfa6b6f5e6c7344e216d4d1976c516633e127"} Jul 01 17:01:52.248853 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:52.248807 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.231252518 podStartE2EDuration="2.248791081s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.538563ms" isStatefulPod=false podCreationTimestamp="2026-07-01 17:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:01:52.245891513 +0000 UTC m=+201.414906447" watchObservedRunningTime="2026-07-01 17:01:52.248791081 +0000 UTC m=+201.417805995" Jul 01 17:01:55.620368 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:01:55.620334 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:02:50.620063 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:02:50.620025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:02:50.635003 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:02:50.634973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:02:51.449591 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:02:51.449569 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 17:03:31.258816 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:03:31.258792 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:03:31.269491 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:03:31.269463 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:03:31.271161 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:03:31.271139 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:03:31.273488 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:03:31.273470 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 17:03:31.278370 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:03:31.278355 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:06:34.571113 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.571081 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-4tbmz"] Jul 01 17:06:34.574332 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.574317 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.577740 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.577715 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 01 17:06:34.577902 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.577883 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 17:06:34.577988 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.577969 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 17:06:34.578444 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.578429 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-5wwg2\"" Jul 01 17:06:34.593322 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.593292 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-4tbmz"] Jul 01 17:06:34.671524 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.671489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6cbj\" (UniqueName: \"kubernetes.io/projected/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-kube-api-access-s6cbj\") pod \"s3-init-4tbmz\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.671689 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.671574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-model-cache\") pod \"s3-init-4tbmz\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.772935 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.772902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6cbj\" (UniqueName: \"kubernetes.io/projected/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-kube-api-access-s6cbj\") pod \"s3-init-4tbmz\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.773107 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.773000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-model-cache\") pod \"s3-init-4tbmz\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.773364 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.773344 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-model-cache\") pod \"s3-init-4tbmz\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.783912 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.783883 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6cbj\" (UniqueName: \"kubernetes.io/projected/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-kube-api-access-s6cbj\") pod \"s3-init-4tbmz\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " pod="kserve/s3-init-4tbmz" Jul 01 17:06:34.883075 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:34.882993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4tbmz" Jul 01 17:06:35.018237 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:35.018211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-4tbmz"] Jul 01 17:06:35.020738 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:06:35.020712 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcb70bcf_85c2_4ad1_8ec9_7313d6147a80.slice/crio-648bc01a5873d49e0045434580f5a38aec2b11e08544c8336f6baafe5a288db8 WatchSource:0}: Error finding container 648bc01a5873d49e0045434580f5a38aec2b11e08544c8336f6baafe5a288db8: Status 404 returned error can't find the container with id 648bc01a5873d49e0045434580f5a38aec2b11e08544c8336f6baafe5a288db8 Jul 01 17:06:35.022516 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:35.022499 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 17:06:35.280996 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:35.280948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4tbmz" event={"ID":"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80","Type":"ContainerStarted","Data":"648bc01a5873d49e0045434580f5a38aec2b11e08544c8336f6baafe5a288db8"} Jul 01 17:06:39.300396 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:39.300362 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4tbmz" event={"ID":"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80","Type":"ContainerStarted","Data":"f1cf25bf1709c36a25858b96d1ed0346d177c4da3b9ec887bd7f9a34c0cb9ca7"} Jul 01 17:06:41.309853 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:41.309820 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerID="f1cf25bf1709c36a25858b96d1ed0346d177c4da3b9ec887bd7f9a34c0cb9ca7" exitCode=0 Jul 01 17:06:41.310287 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:41.309859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4tbmz" event={"ID":"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80","Type":"ContainerDied","Data":"f1cf25bf1709c36a25858b96d1ed0346d177c4da3b9ec887bd7f9a34c0cb9ca7"} Jul 01 17:06:46.333638 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:46.333604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4tbmz" event={"ID":"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80","Type":"ContainerStarted","Data":"43c4ecf3a1afaf15e04cc26a469910a2944cd56834e1148c3ac33c3741a869ff"} Jul 01 17:06:46.352169 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:06:46.352126 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-4tbmz" podStartSLOduration=2.129094754 podStartE2EDuration="12.35211072s" totalImagesPullingTime="8.237910102s" totalInitContainerRuntime="1.985105864s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:06:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:06:46.350869147 +0000 UTC m=+495.519884078" watchObservedRunningTime="2026-07-01 17:06:46.35211072 +0000 UTC m=+495.521125636" Jul 01 17:07:51.575437 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:51.575348 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerID="43c4ecf3a1afaf15e04cc26a469910a2944cd56834e1148c3ac33c3741a869ff" exitCode=0 Jul 01 17:07:51.575437 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:51.575388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4tbmz" event={"ID":"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80","Type":"ContainerDied","Data":"43c4ecf3a1afaf15e04cc26a469910a2944cd56834e1148c3ac33c3741a869ff"} Jul 01 17:07:52.701972 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.701927 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4tbmz" Jul 01 17:07:52.834746 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.834651 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6cbj\" (UniqueName: \"kubernetes.io/projected/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-kube-api-access-s6cbj\") pod \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " Jul 01 17:07:52.834746 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.834741 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-model-cache\") pod \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\" (UID: \"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80\") " Jul 01 17:07:52.837068 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.837041 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-kube-api-access-s6cbj" (OuterVolumeSpecName: "kube-api-access-s6cbj") pod "bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" (UID: "bcb70bcf-85c2-4ad1-8ec9-7313d6147a80"). InnerVolumeSpecName "kube-api-access-s6cbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:07:52.853644 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.853608 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-model-cache" (OuterVolumeSpecName: "model-cache") pod "bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" (UID: "bcb70bcf-85c2-4ad1-8ec9-7313d6147a80"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:07:52.935796 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.935761 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-model-cache\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:07:52.935796 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:52.935794 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6cbj\" (UniqueName: \"kubernetes.io/projected/bcb70bcf-85c2-4ad1-8ec9-7313d6147a80-kube-api-access-s6cbj\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:07:53.584515 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:53.584490 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4tbmz" Jul 01 17:07:53.584515 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:53.584496 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4tbmz" event={"ID":"bcb70bcf-85c2-4ad1-8ec9-7313d6147a80","Type":"ContainerDied","Data":"648bc01a5873d49e0045434580f5a38aec2b11e08544c8336f6baafe5a288db8"} Jul 01 17:07:53.584708 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:07:53.584532 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="648bc01a5873d49e0045434580f5a38aec2b11e08544c8336f6baafe5a288db8" Jul 01 17:08:06.571939 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.571900 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587"] Jul 01 17:08:06.572326 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.572235 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerName="s3-init" Jul 01 17:08:06.572326 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.572253 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerName="s3-init" Jul 01 17:08:06.572326 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.572273 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerName="download-hf-model" Jul 01 17:08:06.572326 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.572279 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerName="download-hf-model" Jul 01 17:08:06.572451 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.572350 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb70bcf-85c2-4ad1-8ec9-7313d6147a80" containerName="s3-init" Jul 01 17:08:06.574351 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.574335 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.582633 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.582612 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 01 17:08:06.582774 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.582752 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 01 17:08:06.582836 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.582762 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 01 17:08:06.582919 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.582904 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-csvp5\"" Jul 01 17:08:06.583573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.583549 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 01 17:08:06.583679 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.583585 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 01 17:08:06.583679 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.583630 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 17:08:06.595793 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.595773 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587"] Jul 01 17:08:06.657162 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.657127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.657315 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.657180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.657315 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.657206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.657315 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.657240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.657315 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.657260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzrwr\" (UniqueName: \"kubernetes.io/projected/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-api-access-jzrwr\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.758653 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.758612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.758864 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.758662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.758864 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.758716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.758864 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.758743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzrwr\" (UniqueName: \"kubernetes.io/projected/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-api-access-jzrwr\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.758864 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.758798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.759156 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.759132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.759430 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.759405 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.759527 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.759463 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.761442 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.761413 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.774623 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.774601 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzrwr\" (UniqueName: \"kubernetes.io/projected/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-api-access-jzrwr\") pod \"isvc-xgboost-graph-predictor-555fc69d56-2p587\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.884101 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.884016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:06.890649 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.890621 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww"] Jul 01 17:08:06.893701 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.893636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:06.897443 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.897420 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 01 17:08:06.897554 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.897491 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 01 17:08:06.913610 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:06.913586 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww"] Jul 01 17:08:07.021802 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.021780 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587"] Jul 01 17:08:07.023795 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:08:07.023768 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6adae20_bf8f_43a9_9596_0b15cc94ccbd.slice/crio-3f38e25836a3c78fbf108bc0ba622d4b7b9d81a2c6a79f8bf6fb8664602a3435 WatchSource:0}: Error finding container 3f38e25836a3c78fbf108bc0ba622d4b7b9d81a2c6a79f8bf6fb8664602a3435: Status 404 returned error can't find the container with id 3f38e25836a3c78fbf108bc0ba622d4b7b9d81a2c6a79f8bf6fb8664602a3435 Jul 01 17:08:07.062082 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.062057 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc4nt\" (UniqueName: \"kubernetes.io/projected/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-api-access-gc4nt\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.062219 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.062114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4826e838-fb4b-49d9-9674-2e9f82278e7c-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.062267 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.062229 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.062303 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.062280 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.062303 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.062296 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.163600 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.163510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.163600 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.163568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.163600 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.163597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.163866 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.163630 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc4nt\" (UniqueName: \"kubernetes.io/projected/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-api-access-gc4nt\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.163866 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.163663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4826e838-fb4b-49d9-9674-2e9f82278e7c-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.163866 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:08:07.163680 2580 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 01 17:08:07.163866 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:08:07.163749 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls podName:4826e838-fb4b-49d9-9674-2e9f82278e7c nodeName:}" failed. No retries permitted until 2026-07-01 17:08:07.66372805 +0000 UTC m=+576.832742945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls") pod "isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" (UID: "4826e838-fb4b-49d9-9674-2e9f82278e7c") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 01 17:08:07.164216 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.164195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4826e838-fb4b-49d9-9674-2e9f82278e7c-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.164431 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.164409 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.164431 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.164420 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.186678 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.186650 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc4nt\" (UniqueName: \"kubernetes.io/projected/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-api-access-gc4nt\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.639595 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.639557 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerStarted","Data":"19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5"} Jul 01 17:08:07.640012 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.639613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerStarted","Data":"3f38e25836a3c78fbf108bc0ba622d4b7b9d81a2c6a79f8bf6fb8664602a3435"} Jul 01 17:08:07.668113 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.668080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.670547 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.670527 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.813223 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.813194 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:07.964204 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:07.964173 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww"] Jul 01 17:08:07.967873 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:08:07.967844 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4826e838_fb4b_49d9_9674_2e9f82278e7c.slice/crio-36956756c925d67e33845ede1ad72cddd909bc92bf2dac338a80c55f211cc018 WatchSource:0}: Error finding container 36956756c925d67e33845ede1ad72cddd909bc92bf2dac338a80c55f211cc018: Status 404 returned error can't find the container with id 36956756c925d67e33845ede1ad72cddd909bc92bf2dac338a80c55f211cc018 Jul 01 17:08:08.644872 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:08.644835 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerStarted","Data":"9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf"} Jul 01 17:08:08.645260 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:08.644905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerStarted","Data":"36956756c925d67e33845ede1ad72cddd909bc92bf2dac338a80c55f211cc018"} Jul 01 17:08:11.660043 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:11.660009 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerID="19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5" exitCode=0 Jul 01 17:08:11.660420 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:11.660081 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerDied","Data":"19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5"} Jul 01 17:08:12.666868 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:12.666828 2580 generic.go:356] "Generic (PLEG): container finished" podID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerID="9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf" exitCode=0 Jul 01 17:08:12.666868 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:12.666872 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerDied","Data":"9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf"} Jul 01 17:08:28.756568 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:28.756530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerStarted","Data":"a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff"} Jul 01 17:08:36.961625 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:36.961531 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:08:36.963170 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:36.963144 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:08:36.972462 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:36.972437 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:08:36.974898 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:36.974788 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:08:37.801103 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:37.801062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerStarted","Data":"21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1"} Jul 01 17:08:39.812337 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.812298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerStarted","Data":"2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2"} Jul 01 17:08:39.812845 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.812380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:39.812845 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.812525 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:39.814198 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.814163 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:08:39.814530 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.814511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerStarted","Data":"44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a"} Jul 01 17:08:39.814659 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.814649 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:39.831794 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.831754 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podStartSLOduration=3.528867711 podStartE2EDuration="33.831736895s" totalImagesPullingTime="26.659623461s" totalInitContainerRuntime="3.643245723s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:08:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:08:39.830652134 +0000 UTC m=+608.999667069" watchObservedRunningTime="2026-07-01 17:08:39.831736895 +0000 UTC m=+609.000751813" Jul 01 17:08:39.851468 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:39.851428 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podStartSLOduration=2.471553905 podStartE2EDuration="33.851415356s" totalImagesPullingTime="27.584138558s" totalInitContainerRuntime="3.795722893s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:08:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:08:39.84948972 +0000 UTC m=+609.018504656" watchObservedRunningTime="2026-07-01 17:08:39.851415356 +0000 UTC m=+609.020430270" Jul 01 17:08:40.819016 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:40.818952 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:08:40.819457 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:40.819074 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:40.820359 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:40.820328 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:08:41.822258 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:41.822215 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:08:45.824021 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:45.823984 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:08:45.824628 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:45.824596 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:08:46.826551 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:46.826524 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:08:46.827007 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:46.826979 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:08:55.825457 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:55.825377 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:08:56.827018 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:08:56.826951 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:09:05.825188 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:05.825149 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:09:06.827166 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:06.827125 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:09:15.824643 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:15.824598 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:09:16.827466 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:16.827423 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:09:25.825360 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:25.825315 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:09:26.827302 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:26.827266 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:09:35.825070 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:35.825031 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused" Jul 01 17:09:36.827892 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:36.827856 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:09:45.825997 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:09:45.825942 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:10:16.428597 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.428562 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587"] Jul 01 17:10:16.429139 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.428809 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" containerID="cri-o://21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1" gracePeriod=30 Jul 01 17:10:16.429139 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.428864 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kube-rbac-proxy" containerID="cri-o://44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a" gracePeriod=30 Jul 01 17:10:16.513649 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.513615 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww"] Jul 01 17:10:16.513863 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.513833 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" containerID="cri-o://a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff" gracePeriod=30 Jul 01 17:10:16.514031 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.513879 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kube-rbac-proxy" containerID="cri-o://2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2" gracePeriod=30 Jul 01 17:10:16.822680 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.822643 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused" Jul 01 17:10:16.827844 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:16.827815 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused" Jul 01 17:10:17.207118 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:17.207032 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerID="44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a" exitCode=2 Jul 01 17:10:17.207118 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:17.207106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerDied","Data":"44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a"} Jul 01 17:10:17.209126 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:17.209095 2580 generic.go:356] "Generic (PLEG): container finished" podID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerID="2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2" exitCode=2 Jul 01 17:10:17.209225 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:17.209162 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerDied","Data":"2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2"} Jul 01 17:10:19.767893 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.767864 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:10:19.869103 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869027 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-proxy-tls\") pod \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " Jul 01 17:10:19.869103 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869090 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kserve-provision-location\") pod \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " Jul 01 17:10:19.869299 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869134 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-cabundle-cert\") pod \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " Jul 01 17:10:19.869299 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869160 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-rbac-proxy-sar-config\") pod \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " Jul 01 17:10:19.869299 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869201 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzrwr\" (UniqueName: \"kubernetes.io/projected/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-api-access-jzrwr\") pod \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\" (UID: \"c6adae20-bf8f-43a9-9596-0b15cc94ccbd\") " Jul 01 17:10:19.869504 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869464 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c6adae20-bf8f-43a9-9596-0b15cc94ccbd" (UID: "c6adae20-bf8f-43a9-9596-0b15cc94ccbd"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:10:19.869504 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869477 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c6adae20-bf8f-43a9-9596-0b15cc94ccbd" (UID: "c6adae20-bf8f-43a9-9596-0b15cc94ccbd"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:10:19.869660 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.869551 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c6adae20-bf8f-43a9-9596-0b15cc94ccbd" (UID: "c6adae20-bf8f-43a9-9596-0b15cc94ccbd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:10:19.871345 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.871324 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c6adae20-bf8f-43a9-9596-0b15cc94ccbd" (UID: "c6adae20-bf8f-43a9-9596-0b15cc94ccbd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:10:19.871498 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.871475 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-api-access-jzrwr" (OuterVolumeSpecName: "kube-api-access-jzrwr") pod "c6adae20-bf8f-43a9-9596-0b15cc94ccbd" (UID: "c6adae20-bf8f-43a9-9596-0b15cc94ccbd"). InnerVolumeSpecName "kube-api-access-jzrwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:10:19.970775 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.970747 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-cabundle-cert\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:19.970775 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.970774 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:19.970920 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.970784 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzrwr\" (UniqueName: \"kubernetes.io/projected/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kube-api-access-jzrwr\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:19.970920 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.970794 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-proxy-tls\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:19.970920 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:19.970802 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c6adae20-bf8f-43a9-9596-0b15cc94ccbd-kserve-provision-location\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:20.221943 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.221863 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerID="21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1" exitCode=0 Jul 01 17:10:20.221943 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.221909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerDied","Data":"21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1"} Jul 01 17:10:20.222128 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.221919 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" Jul 01 17:10:20.222128 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.221993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587" event={"ID":"c6adae20-bf8f-43a9-9596-0b15cc94ccbd","Type":"ContainerDied","Data":"3f38e25836a3c78fbf108bc0ba622d4b7b9d81a2c6a79f8bf6fb8664602a3435"} Jul 01 17:10:20.222128 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.222030 2580 scope.go:117] "RemoveContainer" containerID="44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a" Jul 01 17:10:20.230999 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.230981 2580 scope.go:117] "RemoveContainer" containerID="21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1" Jul 01 17:10:20.238526 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.238508 2580 scope.go:117] "RemoveContainer" containerID="19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5" Jul 01 17:10:20.244195 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.244177 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587"] Jul 01 17:10:20.245951 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.245937 2580 scope.go:117] "RemoveContainer" containerID="44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a" Jul 01 17:10:20.246240 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:10:20.246221 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a\": container with ID starting with 44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a not found: ID does not exist" containerID="44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a" Jul 01 17:10:20.246285 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.246249 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a"} err="failed to get container status \"44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a\": rpc error: code = NotFound desc = could not find container \"44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a\": container with ID starting with 44bfc3dddf52adae5c7dea2f6c7e2bc7133a12d393e9e50cf91ad01410be929a not found: ID does not exist" Jul 01 17:10:20.246285 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.246265 2580 scope.go:117] "RemoveContainer" containerID="21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1" Jul 01 17:10:20.246557 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:10:20.246526 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1\": container with ID starting with 21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1 not found: ID does not exist" containerID="21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1" Jul 01 17:10:20.246599 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.246564 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1"} err="failed to get container status \"21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1\": rpc error: code = NotFound desc = could not find container \"21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1\": container with ID starting with 21e949d677ffce7a1570ea9fceae31cf6f8ce5ac94873f9edc41b41dfdc650c1 not found: ID does not exist" Jul 01 17:10:20.246599 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.246580 2580 scope.go:117] "RemoveContainer" containerID="19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5" Jul 01 17:10:20.246766 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:10:20.246752 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5\": container with ID starting with 19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5 not found: ID does not exist" containerID="19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5" Jul 01 17:10:20.246807 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.246771 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5"} err="failed to get container status \"19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5\": rpc error: code = NotFound desc = could not find container \"19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5\": container with ID starting with 19d8c54cff875756bb8d8e741f02d6e23d4bb56ed37dabbe51cc869808cd46d5 not found: ID does not exist" Jul 01 17:10:20.250503 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.250485 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-2p587"] Jul 01 17:10:20.851088 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.851060 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:10:20.878445 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878417 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4826e838-fb4b-49d9-9674-2e9f82278e7c-kserve-provision-location\") pod \"4826e838-fb4b-49d9-9674-2e9f82278e7c\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " Jul 01 17:10:20.878588 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878466 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-rbac-proxy-sar-config\") pod \"4826e838-fb4b-49d9-9674-2e9f82278e7c\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " Jul 01 17:10:20.878588 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878489 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc4nt\" (UniqueName: \"kubernetes.io/projected/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-api-access-gc4nt\") pod \"4826e838-fb4b-49d9-9674-2e9f82278e7c\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " Jul 01 17:10:20.878588 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878553 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls\") pod \"4826e838-fb4b-49d9-9674-2e9f82278e7c\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " Jul 01 17:10:20.878588 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878582 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-cabundle-cert\") pod \"4826e838-fb4b-49d9-9674-2e9f82278e7c\" (UID: \"4826e838-fb4b-49d9-9674-2e9f82278e7c\") " Jul 01 17:10:20.878804 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878768 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4826e838-fb4b-49d9-9674-2e9f82278e7c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4826e838-fb4b-49d9-9674-2e9f82278e7c" (UID: "4826e838-fb4b-49d9-9674-2e9f82278e7c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:10:20.878917 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878794 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4826e838-fb4b-49d9-9674-2e9f82278e7c" (UID: "4826e838-fb4b-49d9-9674-2e9f82278e7c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:10:20.879010 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.878993 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4826e838-fb4b-49d9-9674-2e9f82278e7c" (UID: "4826e838-fb4b-49d9-9674-2e9f82278e7c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:10:20.880804 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.880777 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4826e838-fb4b-49d9-9674-2e9f82278e7c" (UID: "4826e838-fb4b-49d9-9674-2e9f82278e7c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:10:20.880804 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.880793 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-api-access-gc4nt" (OuterVolumeSpecName: "kube-api-access-gc4nt") pod "4826e838-fb4b-49d9-9674-2e9f82278e7c" (UID: "4826e838-fb4b-49d9-9674-2e9f82278e7c"). InnerVolumeSpecName "kube-api-access-gc4nt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:10:20.979322 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.979301 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4826e838-fb4b-49d9-9674-2e9f82278e7c-proxy-tls\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:20.979322 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.979322 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-cabundle-cert\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:20.979449 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.979331 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4826e838-fb4b-49d9-9674-2e9f82278e7c-kserve-provision-location\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:20.979449 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.979341 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:20.979449 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:20.979350 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc4nt\" (UniqueName: \"kubernetes.io/projected/4826e838-fb4b-49d9-9674-2e9f82278e7c-kube-api-access-gc4nt\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:10:21.229253 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.229221 2580 generic.go:356] "Generic (PLEG): container finished" podID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerID="a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff" exitCode=0 Jul 01 17:10:21.229421 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.229292 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" Jul 01 17:10:21.229421 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.229304 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerDied","Data":"a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff"} Jul 01 17:10:21.229421 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.229370 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" event={"ID":"4826e838-fb4b-49d9-9674-2e9f82278e7c","Type":"ContainerDied","Data":"36956756c925d67e33845ede1ad72cddd909bc92bf2dac338a80c55f211cc018"} Jul 01 17:10:21.229421 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.229412 2580 scope.go:117] "RemoveContainer" containerID="2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2" Jul 01 17:10:21.238344 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.238329 2580 scope.go:117] "RemoveContainer" containerID="a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff" Jul 01 17:10:21.245304 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.245286 2580 scope.go:117] "RemoveContainer" containerID="9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf" Jul 01 17:10:21.253147 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253130 2580 scope.go:117] "RemoveContainer" containerID="2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2" Jul 01 17:10:21.253416 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:10:21.253392 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2\": container with ID starting with 2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2 not found: ID does not exist" containerID="2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2" Jul 01 17:10:21.253583 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253426 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2"} err="failed to get container status \"2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2\": rpc error: code = NotFound desc = could not find container \"2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2\": container with ID starting with 2021687b35edfbad969f47d0545d6d5b8a7e395519d8eb9c2ea1270d020f0ce2 not found: ID does not exist" Jul 01 17:10:21.253583 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253448 2580 scope.go:117] "RemoveContainer" containerID="a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff" Jul 01 17:10:21.253706 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253617 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww"] Jul 01 17:10:21.253706 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:10:21.253681 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff\": container with ID starting with a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff not found: ID does not exist" containerID="a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff" Jul 01 17:10:21.253809 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253712 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff"} err="failed to get container status \"a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff\": rpc error: code = NotFound desc = could not find container \"a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff\": container with ID starting with a892c209586d6b1a43293f1231a6f99c6265af13cedae9a5e256c7b2867604ff not found: ID does not exist" Jul 01 17:10:21.253809 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253737 2580 scope.go:117] "RemoveContainer" containerID="9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf" Jul 01 17:10:21.253981 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:10:21.253944 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf\": container with ID starting with 9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf not found: ID does not exist" containerID="9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf" Jul 01 17:10:21.254026 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.253987 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf"} err="failed to get container status \"9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf\": rpc error: code = NotFound desc = could not find container \"9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf\": container with ID starting with 9956091e16c51b8e9492370c7dc225fbfb2d9bca79130082be7ba7dd6eb09ddf not found: ID does not exist" Jul 01 17:10:21.256365 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.256347 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww"] Jul 01 17:10:21.341210 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.341190 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" path="/var/lib/kubelet/pods/4826e838-fb4b-49d9-9674-2e9f82278e7c/volumes" Jul 01 17:10:21.341647 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.341634 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" path="/var/lib/kubelet/pods/c6adae20-bf8f-43a9-9596-0b15cc94ccbd/volumes" Jul 01 17:10:21.819815 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:10:21.819765 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6c8478f9b5-4vpww" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": context deadline exceeded" Jul 01 17:13:36.992546 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:13:36.992479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:13:36.993028 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:13:36.992543 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:13:36.999121 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:13:36.999102 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:13:36.999244 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:13:36.999208 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:18:37.016720 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:18:37.016691 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:18:37.018942 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:18:37.018369 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:18:37.023855 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:18:37.023834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:18:37.025610 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:18:37.025589 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:23:37.041221 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:23:37.041090 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:23:37.045159 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:23:37.043569 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:23:37.050338 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:23:37.050319 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:23:37.052547 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:23:37.052530 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:28:37.069428 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:28:37.069308 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:28:37.071097 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:28:37.071076 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:28:37.076523 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:28:37.076501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:28:37.078423 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:28:37.078406 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:33:37.094447 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:33:37.094346 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:33:37.098219 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:33:37.096063 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:33:37.101406 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:33:37.101386 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:33:37.103302 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:33:37.103281 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:38:37.120003 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:38:37.119886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:38:37.124185 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:38:37.123113 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:38:37.127325 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:38:37.127306 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:38:37.130337 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:38:37.130315 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:43:37.143455 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:43:37.143344 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:43:37.148582 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:43:37.148565 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:43:37.150041 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:43:37.150025 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:43:37.154661 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:43:37.154644 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x9llh_bd251530-5dc0-48cf-9351-4b69edd423bc/ovn-acl-logging/0.log" Jul 01 17:47:27.234921 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.234886 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c4mjn/must-gather-fcmdl"] Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235184 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="storage-initializer" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235195 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="storage-initializer" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235202 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235207 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235215 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kube-rbac-proxy" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235220 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kube-rbac-proxy" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235226 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kube-rbac-proxy" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235231 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kube-rbac-proxy" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235242 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="storage-initializer" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235247 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="storage-initializer" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235253 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235260 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235303 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kserve-container" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235309 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4826e838-fb4b-49d9-9674-2e9f82278e7c" containerName="kube-rbac-proxy" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235318 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kube-rbac-proxy" Jul 01 17:47:27.235378 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.235324 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6adae20-bf8f-43a9-9596-0b15cc94ccbd" containerName="kserve-container" Jul 01 17:47:27.238091 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.237932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.241035 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.241017 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c4mjn\"/\"default-dockercfg-nl2p7\"" Jul 01 17:47:27.243164 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.243141 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c4mjn\"/\"kube-root-ca.crt\"" Jul 01 17:47:27.243260 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.243187 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c4mjn\"/\"openshift-service-ca.crt\"" Jul 01 17:47:27.247584 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.247550 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c4mjn/must-gather-fcmdl"] Jul 01 17:47:27.348273 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.348251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-must-gather-output\") pod \"must-gather-fcmdl\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.348364 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.348318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plkdb\" (UniqueName: \"kubernetes.io/projected/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-kube-api-access-plkdb\") pod \"must-gather-fcmdl\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.449569 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.449548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plkdb\" (UniqueName: \"kubernetes.io/projected/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-kube-api-access-plkdb\") pod \"must-gather-fcmdl\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.449649 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.449581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-must-gather-output\") pod \"must-gather-fcmdl\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.449838 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.449824 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-must-gather-output\") pod \"must-gather-fcmdl\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.459052 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.459031 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plkdb\" (UniqueName: \"kubernetes.io/projected/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-kube-api-access-plkdb\") pod \"must-gather-fcmdl\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.547026 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.546980 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:47:27.671115 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.671093 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c4mjn/must-gather-fcmdl"] Jul 01 17:47:27.673056 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:47:27.673024 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81a8b3eb_8e50_4bad_9b02_e00e5c976bca.slice/crio-5cb05c96e63e018ba4029692c82b9c0da4bd363d433fa33ca698de69572f4357 WatchSource:0}: Error finding container 5cb05c96e63e018ba4029692c82b9c0da4bd363d433fa33ca698de69572f4357: Status 404 returned error can't find the container with id 5cb05c96e63e018ba4029692c82b9c0da4bd363d433fa33ca698de69572f4357 Jul 01 17:47:27.674881 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:27.674862 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 17:47:28.263314 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:28.263284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" event={"ID":"81a8b3eb-8e50-4bad-9b02-e00e5c976bca","Type":"ContainerStarted","Data":"5cb05c96e63e018ba4029692c82b9c0da4bd363d433fa33ca698de69572f4357"} Jul 01 17:47:34.292093 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:34.291764 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" event={"ID":"81a8b3eb-8e50-4bad-9b02-e00e5c976bca","Type":"ContainerStarted","Data":"ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc"} Jul 01 17:47:34.292093 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:34.291823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" event={"ID":"81a8b3eb-8e50-4bad-9b02-e00e5c976bca","Type":"ContainerStarted","Data":"c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b"} Jul 01 17:47:34.309876 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:34.309825 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" podStartSLOduration=0.972133742 podStartE2EDuration="7.309810882s" totalImagesPullingTime="6.33767714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:47:34.308399664 +0000 UTC m=+2943.477414589" watchObservedRunningTime="2026-07-01 17:47:34.309810882 +0000 UTC m=+2943.478825795" Jul 01 17:47:57.387358 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:57.387327 2580 generic.go:356] "Generic (PLEG): container finished" podID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerID="c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b" exitCode=0 Jul 01 17:47:57.387709 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:57.387384 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" event={"ID":"81a8b3eb-8e50-4bad-9b02-e00e5c976bca","Type":"ContainerDied","Data":"c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b"} Jul 01 17:47:57.387709 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:57.387660 2580 scope.go:117] "RemoveContainer" containerID="c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b" Jul 01 17:47:57.994531 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:47:57.994484 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-c4mjn_must-gather-fcmdl_81a8b3eb-8e50-4bad-9b02-e00e5c976bca/gather/0.log" Jul 01 17:48:01.952499 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:01.952467 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-whqr9_f9b9fb6e-3231-44c6-8960-0ef674cd11d9/global-pull-secret-syncer/0.log" Jul 01 17:48:02.241514 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:02.241488 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m4cmr_dc7892fa-6fbd-4939-b1e4-55b84f5c6369/konnectivity-agent/0.log" Jul 01 17:48:02.503635 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:02.503565 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-168.ec2.internal_0ecf426856d647352e1707c5d6b53ca4/haproxy/0.log" Jul 01 17:48:03.507735 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.507707 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-c4mjn/must-gather-fcmdl"] Jul 01 17:48:03.508113 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.507863 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="copy" containerID="cri-o://ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc" gracePeriod=2 Jul 01 17:48:03.512836 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.512803 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-c4mjn/must-gather-fcmdl"] Jul 01 17:48:03.730505 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.730485 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-c4mjn_must-gather-fcmdl_81a8b3eb-8e50-4bad-9b02-e00e5c976bca/copy/0.log" Jul 01 17:48:03.732335 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.732317 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:48:03.734671 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:48:03.734650 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fcmdl\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-c4mjn\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:48:03.844736 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.844689 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plkdb\" (UniqueName: \"kubernetes.io/projected/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-kube-api-access-plkdb\") pod \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " Jul 01 17:48:03.844818 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.844740 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-must-gather-output\") pod \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\" (UID: \"81a8b3eb-8e50-4bad-9b02-e00e5c976bca\") " Jul 01 17:48:03.846252 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.846228 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "81a8b3eb-8e50-4bad-9b02-e00e5c976bca" (UID: "81a8b3eb-8e50-4bad-9b02-e00e5c976bca"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:48:03.846756 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.846731 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-kube-api-access-plkdb" (OuterVolumeSpecName: "kube-api-access-plkdb") pod "81a8b3eb-8e50-4bad-9b02-e00e5c976bca" (UID: "81a8b3eb-8e50-4bad-9b02-e00e5c976bca"). InnerVolumeSpecName "kube-api-access-plkdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:48:03.945084 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.945067 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plkdb\" (UniqueName: \"kubernetes.io/projected/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-kube-api-access-plkdb\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:48:03.945084 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:03.945085 2580 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81a8b3eb-8e50-4bad-9b02-e00e5c976bca-must-gather-output\") on node \"ip-10-0-140-168.ec2.internal\" DevicePath \"\"" Jul 01 17:48:04.413868 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.413847 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-c4mjn_must-gather-fcmdl_81a8b3eb-8e50-4bad-9b02-e00e5c976bca/copy/0.log" Jul 01 17:48:04.414169 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.414146 2580 generic.go:356] "Generic (PLEG): container finished" podID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerID="ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc" exitCode=143 Jul 01 17:48:04.414229 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.414191 2580 scope.go:117] "RemoveContainer" containerID="ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc" Jul 01 17:48:04.414229 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.414198 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:48:04.418459 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:48:04.418423 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fcmdl\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-c4mjn\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:48:04.421936 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.421919 2580 scope.go:117] "RemoveContainer" containerID="c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b" Jul 01 17:48:04.424387 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:48:04.424361 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-fcmdl\" is forbidden: User \"system:node:ip-10-0-140-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-c4mjn\": no relationship found between node 'ip-10-0-140-168.ec2.internal' and this object" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" pod="openshift-must-gather-c4mjn/must-gather-fcmdl" Jul 01 17:48:04.433146 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.433131 2580 scope.go:117] "RemoveContainer" containerID="ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc" Jul 01 17:48:04.433380 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:48:04.433362 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc\": container with ID starting with ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc not found: ID does not exist" containerID="ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc" Jul 01 17:48:04.433450 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.433390 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc"} err="failed to get container status \"ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc\": rpc error: code = NotFound desc = could not find container \"ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc\": container with ID starting with ec258240250196264b9c7889f90ca4c96cb167118c91779d2ca8e6890b4b47fc not found: ID does not exist" Jul 01 17:48:04.433450 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.433413 2580 scope.go:117] "RemoveContainer" containerID="c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b" Jul 01 17:48:04.433660 ip-10-0-140-168 kubenswrapper[2580]: E0701 17:48:04.433643 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b\": container with ID starting with c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b not found: ID does not exist" containerID="c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b" Jul 01 17:48:04.433699 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:04.433665 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b"} err="failed to get container status \"c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b\": rpc error: code = NotFound desc = could not find container \"c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b\": container with ID starting with c01b10864403ad5f36346dada504f1f101e7b02cd511361be7aad02d51c19c3b not found: ID does not exist" Jul 01 17:48:05.341333 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:05.341301 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" path="/var/lib/kubelet/pods/81a8b3eb-8e50-4bad-9b02-e00e5c976bca/volumes" Jul 01 17:48:07.391057 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.391033 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kgmwl_a9493dd9-f5d6-4afe-b3d9-88c1a8504905/node-exporter/0.log" Jul 01 17:48:07.420746 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.420725 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kgmwl_a9493dd9-f5d6-4afe-b3d9-88c1a8504905/kube-rbac-proxy/0.log" Jul 01 17:48:07.448676 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.448655 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kgmwl_a9493dd9-f5d6-4afe-b3d9-88c1a8504905/init-textfile/0.log" Jul 01 17:48:07.478824 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.478801 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-2kqml_3da6bdfd-df66-465c-ae5d-af1146636a5b/kube-rbac-proxy-main/0.log" Jul 01 17:48:07.507442 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.507419 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-2kqml_3da6bdfd-df66-465c-ae5d-af1146636a5b/kube-rbac-proxy-self/0.log" Jul 01 17:48:07.535039 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.535014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-2kqml_3da6bdfd-df66-465c-ae5d-af1146636a5b/openshift-state-metrics/0.log" Jul 01 17:48:07.582007 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.581985 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/prometheus/0.log" Jul 01 17:48:07.609337 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.609291 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/config-reloader/0.log" Jul 01 17:48:07.638949 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.638927 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/thanos-sidecar/0.log" Jul 01 17:48:07.665730 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.665709 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/kube-rbac-proxy-web/0.log" Jul 01 17:48:07.694281 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.694264 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/kube-rbac-proxy/0.log" Jul 01 17:48:07.719552 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.719532 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/kube-rbac-proxy-thanos/0.log" Jul 01 17:48:07.754517 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.754499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3f0893a6-c147-497e-b662-88d662ef5f3e/init-config-reloader/0.log" Jul 01 17:48:07.801232 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.801206 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-9s8mk_2c8fc67a-98cb-429a-92da-2c4237b0b471/prometheus-operator/0.log" Jul 01 17:48:07.833735 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.833718 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-9s8mk_2c8fc67a-98cb-429a-92da-2c4237b0b471/kube-rbac-proxy/0.log" Jul 01 17:48:07.911309 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.911278 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56f867fbc8-rhp4n_caa7ec7b-b150-402a-b3a5-ab4329132793/telemeter-client/0.log" Jul 01 17:48:07.952032 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.952008 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56f867fbc8-rhp4n_caa7ec7b-b150-402a-b3a5-ab4329132793/reload/0.log" Jul 01 17:48:07.996118 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:07.996099 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56f867fbc8-rhp4n_caa7ec7b-b150-402a-b3a5-ab4329132793/kube-rbac-proxy/0.log" Jul 01 17:48:09.450817 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.450784 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-qq4bf_6e2d57fd-7a5b-4e1a-97f4-56edf0d181f2/networking-console-plugin/0.log" Jul 01 17:48:09.718098 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718038 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf"] Jul 01 17:48:09.718314 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718303 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="gather" Jul 01 17:48:09.718356 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718315 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="gather" Jul 01 17:48:09.718356 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718324 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="copy" Jul 01 17:48:09.718356 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718329 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="copy" Jul 01 17:48:09.718443 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718383 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="gather" Jul 01 17:48:09.718443 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.718391 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a8b3eb-8e50-4bad-9b02-e00e5c976bca" containerName="copy" Jul 01 17:48:09.720181 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.720166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.723048 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.723029 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-76kqn\"/\"default-dockercfg-rgc45\"" Jul 01 17:48:09.723862 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.723843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-76kqn\"/\"openshift-service-ca.crt\"" Jul 01 17:48:09.723984 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.723884 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-76kqn\"/\"kube-root-ca.crt\"" Jul 01 17:48:09.745826 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.745806 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf"] Jul 01 17:48:09.781841 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.781818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwrm8\" (UniqueName: \"kubernetes.io/projected/a13fc1dc-b059-491c-818b-c6538a4cca13-kube-api-access-qwrm8\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.781935 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.781866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-proc\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.781935 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.781902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-sys\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.782040 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.781950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-podres\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.782040 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.782002 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-lib-modules\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882573 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-proc\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882665 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-sys\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882665 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-podres\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882744 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-lib-modules\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882744 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882677 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-proc\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882744 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-sys\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882744 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwrm8\" (UniqueName: \"kubernetes.io/projected/a13fc1dc-b059-491c-818b-c6538a4cca13-kube-api-access-qwrm8\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882860 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882739 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-podres\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.882860 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.882768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a13fc1dc-b059-491c-818b-c6538a4cca13-lib-modules\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:09.892827 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:09.892807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwrm8\" (UniqueName: \"kubernetes.io/projected/a13fc1dc-b059-491c-818b-c6538a4cca13-kube-api-access-qwrm8\") pod \"perf-node-gather-daemonset-2nxcf\" (UID: \"a13fc1dc-b059-491c-818b-c6538a4cca13\") " pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:10.021825 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.021781 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/1.log" Jul 01 17:48:10.028786 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.028761 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-mgmvf_362fbfdc-5a59-4933-b0cf-384af6eba934/console-operator/2.log" Jul 01 17:48:10.029867 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.029843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:10.160896 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.160860 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf"] Jul 01 17:48:10.163137 ip-10-0-140-168 kubenswrapper[2580]: W0701 17:48:10.163108 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda13fc1dc_b059_491c_818b_c6538a4cca13.slice/crio-a368924cfa4327943d67951acafc108fe72cff2ce603a7273aa9c23d588e13b5 WatchSource:0}: Error finding container a368924cfa4327943d67951acafc108fe72cff2ce603a7273aa9c23d588e13b5: Status 404 returned error can't find the container with id a368924cfa4327943d67951acafc108fe72cff2ce603a7273aa9c23d588e13b5 Jul 01 17:48:10.438545 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.438480 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" event={"ID":"a13fc1dc-b059-491c-818b-c6538a4cca13","Type":"ContainerStarted","Data":"97e4ee2b714404d4eb48474e8fcbe51a8ba478f22582e13b2d4f58feef804c60"} Jul 01 17:48:10.438545 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.438529 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" event={"ID":"a13fc1dc-b059-491c-818b-c6538a4cca13","Type":"ContainerStarted","Data":"a368924cfa4327943d67951acafc108fe72cff2ce603a7273aa9c23d588e13b5"} Jul 01 17:48:10.438722 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.438619 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:10.461778 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:10.461737 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" podStartSLOduration=1.461722481 podStartE2EDuration="1.461722481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:48:10.461598529 +0000 UTC m=+2979.630613444" watchObservedRunningTime="2026-07-01 17:48:10.461722481 +0000 UTC m=+2979.630737412" Jul 01 17:48:12.082775 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:12.082744 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8q8qz_e3a7cc17-7e6f-4a93-ba6b-f01760f73a48/dns/0.log" Jul 01 17:48:12.111811 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:12.111784 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8q8qz_e3a7cc17-7e6f-4a93-ba6b-f01760f73a48/kube-rbac-proxy/0.log" Jul 01 17:48:12.340868 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:12.340807 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tqhbs_b6eeb377-78e3-414b-bc82-96ad20197434/dns-node-resolver/0.log" Jul 01 17:48:12.830650 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:12.830623 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-868bcc99fc-n78b7_8d3fddc1-4a19-4605-b4af-481ad7a5ef69/registry/0.log" Jul 01 17:48:12.900885 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:12.900860 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cm2bx_eaf2fa6c-6ef1-466b-a465-32c6339204ba/node-ca/0.log" Jul 01 17:48:14.159752 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:14.159730 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mb9bv_1dbea86d-14fb-4a19-bbd8-50ff3cef1006/serve-healthcheck-canary/0.log" Jul 01 17:48:14.652953 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:14.652902 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-lxtr5_e4cde9da-cbdc-4018-8639-7656ff9a614f/insights-operator/0.log" Jul 01 17:48:14.654918 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:14.654894 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-lxtr5_e4cde9da-cbdc-4018-8639-7656ff9a614f/insights-operator/1.log" Jul 01 17:48:14.843781 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:14.843762 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssfv4_4566659d-c605-480c-8d0b-580e851b62f3/kube-rbac-proxy/0.log" Jul 01 17:48:14.889199 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:14.889181 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssfv4_4566659d-c605-480c-8d0b-580e851b62f3/exporter/0.log" Jul 01 17:48:14.931719 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:14.931658 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssfv4_4566659d-c605-480c-8d0b-580e851b62f3/extractor/0.log" Jul 01 17:48:16.450717 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:16.450692 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-76kqn/perf-node-gather-daemonset-2nxcf" Jul 01 17:48:17.723644 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:17.723620 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-4tbmz_bcb70bcf-85c2-4ad1-8ec9-7313d6147a80/s3-init/0.log" Jul 01 17:48:17.767013 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:17.766989 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-4tbmz_bcb70bcf-85c2-4ad1-8ec9-7313d6147a80/download-hf-model/0.log" Jul 01 17:48:23.777214 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:23.777183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-ctxzh_d9f77bb1-30cd-4c03-b45e-026b3113360a/migrator/0.log" Jul 01 17:48:23.843704 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:23.843675 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-ctxzh_d9f77bb1-30cd-4c03-b45e-026b3113360a/graceful-termination/0.log" Jul 01 17:48:25.491582 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.491558 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/kube-multus-additional-cni-plugins/0.log" Jul 01 17:48:25.546405 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.546334 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/egress-router-binary-copy/0.log" Jul 01 17:48:25.596060 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.596038 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/cni-plugins/0.log" Jul 01 17:48:25.638520 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.638499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/bond-cni-plugin/0.log" Jul 01 17:48:25.688764 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.688747 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/routeoverride-cni/0.log" Jul 01 17:48:25.726042 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.726022 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/whereabouts-cni-bincopy/0.log" Jul 01 17:48:25.754733 ip-10-0-140-168 kubenswrapper[2580]: I0701 17:48:25.754714 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bzxl_291e48c3-8b05-4b03-a22d-8daf83066d72/whereabouts-cni/0.log"