I0705 02:16:14.096314 1 cmd.go:250] Using service-serving-cert provided certificates I0705 02:16:14.097412 1 observer_polling.go:159] Starting file observer W0705 02:16:14.123620 1 builder.go:266] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-6fd96f47fd" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0705 02:16:14.560284 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0705 02:16:14.560510 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0705 02:16:14.561005 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0705 02:16:14.561055 1 secure_serving.go:57] Forcing use of http/1.1 only W0705 02:16:14.561083 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0705 02:16:14.561092 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0705 02:16:14.561098 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0705 02:16:14.561101 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0705 02:16:14.561105 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0705 02:16:14.561109 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0705 02:16:14.565027 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes ChunkSizeMiB CloudDualStackNodeIPs DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy PersistentIPsForVirtualization PrivateHostedZoneAWS SetEIPForNLBIngressController VSphereControlPlaneMachineSet VSphereDriverConfiguration VSphereMultiVCenters VSphereStaticIPs ValidatingAdmissionPolicy AWSClusterHostedDNS AdditionalRoutingCapabilities AutomatedEtcdBackup BootcNodeManagement CSIDriverSharedResource ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ConsolePluginContentSecurityPolicy DNSNameResolver DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example GCPClusterHostedDNS GatewayAPI ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MetricsCollectionProfiles MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NodeSwap NutanixMultiSubnets OVNObservability OnClusterBuild OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SignatureStores SigstoreImageVerification TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0705 02:16:14.565079 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "ChunkSizeMiB", "CloudDualStackNodeIPs", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "SetEIPForNLBIngressController", "VSphereControlPlaneMachineSet", "VSphereDriverConfiguration", "VSphereMultiVCenters", "VSphereStaticIPs", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AdditionalRoutingCapabilities", "AutomatedEtcdBackup", "BootcNodeManagement", "CSIDriverSharedResource", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ConsolePluginContentSecurityPolicy", "DNSNameResolver", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "GCPClusterHostedDNS", "GatewayAPI", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MetricsCollectionProfiles", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OnClusterBuild", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SignatureStores", "SigstoreImageVerification", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} E0705 02:16:14.569057 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18bf431e3a1c25ed openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"SetEIPForNLBIngressController\", \"VSphereControlPlaneMachineSet\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"VSphereStaticIPs\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AdditionalRoutingCapabilities\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"CSIDriverSharedResource\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ConsolePluginContentSecurityPolicy\", \"DNSNameResolver\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"GCPClusterHostedDNS\", \"GatewayAPI\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MetricsCollectionProfiles\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OnClusterBuild\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SignatureStores\", \"SigstoreImageVerification\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-07-05 02:16:14.565008877 +0000 UTC m=+0.520303086,LastTimestamp:2026-07-05 02:16:14.565008877 +0000 UTC m=+0.520303086,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" I0705 02:16:14.569250 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController I0705 02:16:14.569295 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0705 02:16:14.569334 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0705 02:16:14.569733 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0705 02:16:14.569766 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0705 02:16:14.569747 1 secure_serving.go:213] Serving securely on [::]:8443 I0705 02:16:14.569767 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0705 02:16:14.569348 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0705 02:16:14.570205 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file W0705 02:16:14.574276 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0705 02:16:14.574303 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0705 02:16:14.574404 1 base_controller.go:67] Waiting for caches to sync for ConfigController I0705 02:16:14.582304 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:16:14.582324 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0705 02:16:14.588144 1 secretconfigobserver.go:119] support secret does not exist I0705 02:16:14.592951 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:16:14.594842 1 recorder.go:156] Pruning old reports every 5h27m17s, max age is 288h0m0s I0705 02:16:14.599319 1 secretconfigobserver.go:119] support secret does not exist I0705 02:16:14.606855 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0705 02:16:14.606859 1 periodic.go:214] Running clusterconfig gatherer I0705 02:16:14.606877 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0705 02:16:14.606895 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0705 02:16:14.606900 1 insightsreport.go:296] Starting report retriever I0705 02:16:14.606904 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0705 02:16:14.606938 1 tasks_processing.go:45] number of workers: 32 I0705 02:16:14.606965 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 02:16:14.606972 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 02:16:14.606981 1 tasks_processing.go:69] worker 19 listening for tasks. I0705 02:16:14.606984 1 tasks_processing.go:69] worker 6 listening for tasks. I0705 02:16:14.606984 1 tasks_processing.go:71] worker 0 working on openshift_machine_api_events task. I0705 02:16:14.606988 1 tasks_processing.go:71] worker 19 working on mutating_webhook_configurations task. I0705 02:16:14.606991 1 tasks_processing.go:69] worker 2 listening for tasks. I0705 02:16:14.606997 1 tasks_processing.go:69] worker 7 listening for tasks. I0705 02:16:14.607000 1 tasks_processing.go:69] worker 3 listening for tasks. I0705 02:16:14.607007 1 tasks_processing.go:69] worker 5 listening for tasks. I0705 02:16:14.607004 1 tasks_processing.go:69] worker 26 listening for tasks. I0705 02:16:14.607012 1 tasks_processing.go:69] worker 8 listening for tasks. I0705 02:16:14.607017 1 tasks_processing.go:69] worker 22 listening for tasks. I0705 02:16:14.607022 1 tasks_processing.go:69] worker 9 listening for tasks. I0705 02:16:14.607025 1 tasks_processing.go:69] worker 20 listening for tasks. I0705 02:16:14.607030 1 tasks_processing.go:69] worker 25 listening for tasks. I0705 02:16:14.607031 1 tasks_processing.go:69] worker 10 listening for tasks. I0705 02:16:14.607034 1 tasks_processing.go:69] worker 21 listening for tasks. I0705 02:16:14.607035 1 tasks_processing.go:69] worker 4 listening for tasks. I0705 02:16:14.607037 1 tasks_processing.go:69] worker 11 listening for tasks. I0705 02:16:14.607043 1 tasks_processing.go:69] worker 23 listening for tasks. I0705 02:16:14.607045 1 tasks_processing.go:69] worker 16 listening for tasks. I0705 02:16:14.607035 1 tasks_processing.go:69] worker 24 listening for tasks. I0705 02:16:14.607049 1 tasks_processing.go:69] worker 13 listening for tasks. I0705 02:16:14.607051 1 tasks_processing.go:69] worker 15 listening for tasks. I0705 02:16:14.607056 1 tasks_processing.go:69] worker 29 listening for tasks. I0705 02:16:14.607049 1 tasks_processing.go:69] worker 14 listening for tasks. I0705 02:16:14.607059 1 tasks_processing.go:69] worker 18 listening for tasks. I0705 02:16:14.607062 1 tasks_processing.go:69] worker 17 listening for tasks. I0705 02:16:14.607065 1 tasks_processing.go:69] worker 28 listening for tasks. I0705 02:16:14.607063 1 tasks_processing.go:69] worker 27 listening for tasks. I0705 02:16:14.607069 1 tasks_processing.go:69] worker 30 listening for tasks. I0705 02:16:14.607069 1 tasks_processing.go:71] worker 1 working on openshift_logging task. I0705 02:16:14.606987 1 tasks_processing.go:69] worker 12 listening for tasks. I0705 02:16:14.607076 1 tasks_processing.go:71] worker 3 working on clusterroles task. I0705 02:16:14.607078 1 tasks_processing.go:71] worker 30 working on install_plans task. I0705 02:16:14.607080 1 tasks_processing.go:71] worker 12 working on operators task. I0705 02:16:14.607086 1 tasks_processing.go:71] worker 5 working on authentication task. I0705 02:16:14.607121 1 tasks_processing.go:71] worker 29 working on storage_classes task. I0705 02:16:14.607081 1 tasks_processing.go:71] worker 11 working on ceph_cluster task. I0705 02:16:14.607309 1 tasks_processing.go:71] worker 26 working on tsdb_status task. I0705 02:16:14.607434 1 tasks_processing.go:71] worker 24 working on image_pruners task. I0705 02:16:14.607473 1 tasks_processing.go:71] worker 10 working on config_maps task. I0705 02:16:14.607514 1 tasks_processing.go:71] worker 15 working on overlapping_namespace_uids task. I0705 02:16:14.607449 1 tasks_processing.go:71] worker 6 working on lokistack task. I0705 02:16:14.607456 1 tasks_processing.go:71] worker 2 working on nodenetworkstates task. I0705 02:16:14.607448 1 tasks_processing.go:71] worker 13 working on node_logs task. I0705 02:16:14.607461 1 tasks_processing.go:71] worker 7 working on openstack_dataplanedeployments task. I0705 02:16:14.607070 1 tasks_processing.go:69] worker 31 listening for tasks. I0705 02:16:14.608034 1 tasks_processing.go:71] worker 31 working on machine_healthchecks task. I0705 02:16:14.607462 1 tasks_processing.go:71] worker 25 working on dvo_metrics task. I0705 02:16:14.607467 1 tasks_processing.go:71] worker 21 working on ingress_certificates task. I0705 02:16:14.607471 1 tasks_processing.go:71] worker 8 working on sap_pods task. I0705 02:16:14.607476 1 tasks_processing.go:71] worker 22 working on sap_datahubs task. I0705 02:16:14.607480 1 tasks_processing.go:71] worker 9 working on image task. I0705 02:16:14.607485 1 tasks_processing.go:71] worker 20 working on oauths task. I0705 02:16:14.607490 1 tasks_processing.go:71] worker 14 working on version task. I0705 02:16:14.607497 1 tasks_processing.go:71] worker 18 working on pod_network_connectivity_checks task. I0705 02:16:14.607496 1 tasks_processing.go:71] worker 4 working on silenced_alerts task. I0705 02:16:14.607503 1 tasks_processing.go:71] worker 17 working on machine_autoscalers task. I0705 02:16:14.607507 1 tasks_processing.go:71] worker 28 working on pdbs task. I0705 02:16:14.607505 1 tasks_processing.go:71] worker 23 working on container_runtime_configs task. I0705 02:16:14.607511 1 tasks_processing.go:71] worker 27 working on sap_config task. I0705 02:16:14.607516 1 tasks_processing.go:71] worker 16 working on cluster_apiserver task. E0705 02:16:14.612903 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.40.21:9091: connect: connection refused I0705 02:16:14.612930 1 tasks_processing.go:71] worker 26 working on proxies task. E0705 02:16:14.612956 1 gather.go:144] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.40.21:9091: connect: connection refused I0705 02:16:14.613009 1 gather.go:181] gatherer "clusterconfig" function "tsdb_status" took 5.564924ms to process 0 records E0705 02:16:14.613270 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.144.49:9094: connect: connection refused I0705 02:16:14.613294 1 tasks_processing.go:71] worker 4 working on machines task. E0705 02:16:14.613307 1 gather.go:144] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.144.49:9094: connect: connection refused I0705 02:16:14.613325 1 gather.go:181] gatherer "clusterconfig" function "silenced_alerts" took 4.1063ms to process 0 records I0705 02:16:14.616793 1 tasks_processing.go:71] worker 1 working on networks task. I0705 02:16:14.616862 1 gather.go:181] gatherer "clusterconfig" function "openshift_logging" took 9.714439ms to process 0 records I0705 02:16:14.616881 1 gather.go:181] gatherer "clusterconfig" function "ceph_cluster" took 9.544509ms to process 0 records I0705 02:16:14.616887 1 gather.go:181] gatherer "clusterconfig" function "lokistack" took 9.241561ms to process 0 records I0705 02:16:14.616888 1 tasks_processing.go:71] worker 11 working on support_secret task. I0705 02:16:14.616895 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkstates" took 9.123684ms to process 0 records I0705 02:16:14.616901 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 8.886622ms to process 0 records I0705 02:16:14.616911 1 tasks_processing.go:71] worker 2 working on machine_config_pools task. I0705 02:16:14.616921 1 tasks_processing.go:71] worker 6 working on cost_management_metrics_configs task. I0705 02:16:14.616930 1 tasks_processing.go:71] worker 8 working on infrastructures task. I0705 02:16:14.616940 1 gather.go:181] gatherer "clusterconfig" function "sap_pods" took 8.567216ms to process 0 records E0705 02:16:14.616951 1 gather.go:144] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0705 02:16:14.616960 1 gather.go:181] gatherer "clusterconfig" function "machine_healthchecks" took 8.890096ms to process 0 records I0705 02:16:14.616968 1 gather.go:181] gatherer "clusterconfig" function "container_runtime_configs" took 6.790949ms to process 0 records I0705 02:16:14.616976 1 tasks_processing.go:71] worker 23 working on openstack_dataplanenodesets task. I0705 02:16:14.616991 1 tasks_processing.go:71] worker 31 working on openstack_version task. I0705 02:16:14.616914 1 tasks_processing.go:71] worker 7 working on aggregated_monitoring_cr_names task. I0705 02:16:14.620863 1 tasks_processing.go:71] worker 22 working on monitoring_persistent_volumes task. I0705 02:16:14.620893 1 gather.go:181] gatherer "clusterconfig" function "sap_datahubs" took 12.41106ms to process 0 records E0705 02:16:14.620912 1 gather.go:144] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0705 02:16:14.620923 1 gather.go:181] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 11.851187ms to process 0 records I0705 02:16:14.620932 1 gather.go:181] gatherer "clusterconfig" function "machine_autoscalers" took 11.262736ms to process 0 records I0705 02:16:14.620938 1 gather.go:181] gatherer "clusterconfig" function "sap_config" took 10.480705ms to process 0 records I0705 02:16:14.620943 1 tasks_processing.go:71] worker 27 working on validating_webhook_configurations task. I0705 02:16:14.621040 1 tasks_processing.go:71] worker 17 working on container_images task. I0705 02:16:14.621061 1 tasks_processing.go:71] worker 18 working on olm_operators task. I0705 02:16:14.635347 1 tasks_processing.go:71] worker 0 working on nodenetworkconfigurationpolicies task. I0705 02:16:14.635346 1 gather.go:181] gatherer "clusterconfig" function "openshift_machine_api_events" took 28.340631ms to process 0 records E0705 02:16:14.635505 1 gather.go:144] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0705 02:16:14.635520 1 gather.go:181] gatherer "clusterconfig" function "machines" took 22.201641ms to process 0 records I0705 02:16:14.635637 1 tasks_processing.go:71] worker 9 working on jaegers task. I0705 02:16:14.635748 1 recorder.go:70] Recording config/image with fingerprint=7d2b68602f0012cbb0828b7b4aa7fe5595651c34b77598867f01738998954bb1 I0705 02:16:14.635771 1 gather.go:181] gatherer "clusterconfig" function "image" took 27.053779ms to process 1 records I0705 02:16:14.635789 1 tasks_processing.go:71] worker 4 working on crds task. I0705 02:16:14.635903 1 tasks_processing.go:71] worker 24 working on machine_sets task. I0705 02:16:14.636093 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0705 02:16:14.636135 1 controller.go:317] The initial operator extension status is healthy I0705 02:16:14.636149 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0705 02:16:14.636158 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0705 02:16:14.636165 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0705 02:16:14.636188 1 controller.go:457] The operator is still being initialized I0705 02:16:14.636202 1 controller.go:482] The operator is healthy I0705 02:16:14.636239 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=f6de32a4d608dc6b4ac0c64650ecbaf413f93be7c732d323cba420b986a057fc I0705 02:16:14.636256 1 gather.go:181] gatherer "clusterconfig" function "image_pruners" took 28.124906ms to process 1 records I0705 02:16:14.636360 1 sca.go:98] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/certificates. Next check is in 8h0m0s I0705 02:16:14.636413 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0705 02:16:14.636460 1 tasks_processing.go:71] worker 16 working on image_registries task. W0705 02:16:14.636515 1 operator.go:288] started I0705 02:16:14.636454 1 recorder.go:70] Recording config/apiserver with fingerprint=7f19ce6bc31fd9fa3e461c1f51f295ae838d5ee9e0e8adfd962a34f91b634c00 I0705 02:16:14.636586 1 gather.go:181] gatherer "clusterconfig" function "cluster_apiserver" took 25.15526ms to process 1 records I0705 02:16:14.636706 1 recorder.go:70] Recording config/proxy with fingerprint=d730575ad6b577038eda225cf491742b717177564a3c51e13230e5b538f4c5d1 I0705 02:16:14.636739 1 gather.go:181] gatherer "clusterconfig" function "proxies" took 22.78562ms to process 1 records I0705 02:16:14.636764 1 gather.go:181] gatherer "clusterconfig" function "machine_config_pools" took 18.856544ms to process 0 records I0705 02:16:14.636814 1 base_controller.go:67] Waiting for caches to sync for LoggingSyncer I0705 02:16:14.636880 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=911aee6179acb18895148b284601b8f0c82621eed0b9ee4915253c113ef44452 I0705 02:16:14.636914 1 tasks_processing.go:71] worker 26 working on openstack_controlplanes task. I0705 02:16:14.636924 1 tasks_processing.go:71] worker 2 working on nodes task. I0705 02:16:14.636942 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=4554fcb518a5df89f5f7f942a2a39b800f1285f651f77152b608463811e908a2 I0705 02:16:14.636914 1 tasks_processing.go:71] worker 29 working on feature_gates task. I0705 02:16:14.636952 1 gather.go:181] gatherer "clusterconfig" function "storage_classes" took 28.658159ms to process 2 records I0705 02:16:14.637126 1 tasks_processing.go:71] worker 5 working on certificate_signing_requests task. I0705 02:16:14.637424 1 recorder.go:70] Recording config/authentication with fingerprint=a1af1ba033eea301d73d4c2fc122f795bde14eb5b22b3c86ce2a03cad9fb5d84 I0705 02:16:14.637449 1 gather.go:181] gatherer "clusterconfig" function "authentication" took 28.858923ms to process 1 records I0705 02:16:14.637461 1 gather.go:181] gatherer "clusterconfig" function "node_logs" took 28.078838ms to process 0 records I0705 02:16:14.637470 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 19.04995ms to process 0 records I0705 02:16:14.637477 1 gather.go:181] gatherer "clusterconfig" function "openstack_version" took 19.407238ms to process 0 records I0705 02:16:14.637482 1 gather.go:181] gatherer "clusterconfig" function "cost_management_metrics_configs" took 19.684918ms to process 0 records I0705 02:16:14.637490 1 tasks_processing.go:71] worker 23 working on storage_cluster task. I0705 02:16:14.637637 1 tasks_processing.go:71] worker 1 working on ingress task. I0705 02:16:14.637677 1 tasks_processing.go:71] worker 6 working on metrics task. I0705 02:16:14.637712 1 tasks_processing.go:71] worker 13 working on service_accounts task. I0705 02:16:14.637762 1 tasks_processing.go:71] worker 31 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0705 02:16:14.637910 1 recorder.go:70] Recording config/network with fingerprint=d941f42129bb7f586c3fe7ef9b7b55fe253b03adcdc3d75a4b8aa153dbaf6b85 I0705 02:16:14.637929 1 gather.go:181] gatherer "clusterconfig" function "networks" took 19.985283ms to process 1 records I0705 02:16:14.638132 1 tasks_processing.go:71] worker 8 working on schedulers task. I0705 02:16:14.639758 1 recorder.go:70] Recording config/infrastructure with fingerprint=9b93b76ded28e94b0602cad58fb0dc175812cd9a278720926c62c10c5e0d05a5 I0705 02:16:14.639817 1 gather.go:181] gatherer "clusterconfig" function "infrastructures" took 20.444795ms to process 1 records I0705 02:16:14.639907 1 tasks_processing.go:71] worker 19 working on machine_configs task. I0705 02:16:14.640098 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=dd3595c96498502bf717a815cbbac50766150824dea10b764b5ded3b5db54cc9 I0705 02:16:14.640151 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=b64783eedafc91a7a8a7d0af48147b0b9308e970a2af12a2e58d50bd005a1460 I0705 02:16:14.640190 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=4fc195dabf3a37656ab111d864fb035c98360ac91d42babdc91a68b5c7a39642 I0705 02:16:14.640199 1 gather.go:181] gatherer "clusterconfig" function "mutating_webhook_configurations" took 32.606064ms to process 3 records I0705 02:16:14.640479 1 tasks_processing.go:71] worker 20 working on operators_pods_and_events task. I0705 02:16:14.640782 1 recorder.go:70] Recording config/oauth with fingerprint=498b0438dc7177d740e4ef9fb9626a5033661937bca341c51d66cade233ea457 I0705 02:16:14.640799 1 gather.go:181] gatherer "clusterconfig" function "oauths" took 30.962767ms to process 1 records I0705 02:16:14.640950 1 recorder.go:70] Recording config/pdbs/openshift-console/console with fingerprint=f9cd3c9eb451515121a78584cfb27e202d64f4d9be4d7d4e8b04a567a744fe0a I0705 02:16:14.640991 1 recorder.go:70] Recording config/pdbs/openshift-console/downloads with fingerprint=7491164678fb56646d45aaa7494a2ea66e3bc7d75abeb9df50464359bfa23e8f I0705 02:16:14.641019 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=a08b41050e17313c3c289427eadb8c67e44a9984520fc262d316747af780da7e I0705 02:16:14.641048 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=b93f44592b4e520459efa621a1d6cb12654ac8528ceba77f2e4fb2337f51c62d I0705 02:16:14.641081 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=0e214d199cee178c2843d5edd9c0c122ccff39027c2f1406881e2825380771aa I0705 02:16:14.641116 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=1a9cf9d0e6f2c05386d756b322a4e8b08f0a99bb529dedafc83dd10504b129b2 I0705 02:16:14.641132 1 tasks_processing.go:71] worker 28 working on active_alerts task. E0705 02:16:14.641330 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.40.21:9091: connect: connection refused I0705 02:16:14.641267 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=5e94ec790b2d4b091f06c5856203e672d1d7fa7b65d5dfa406c0f63fc437b78f I0705 02:16:14.641443 1 gather.go:181] gatherer "clusterconfig" function "pdbs" took 30.436754ms to process 7 records E0705 02:16:14.641454 1 gather.go:144] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.40.21:9091: connect: connection refused I0705 02:16:14.641471 1 gather.go:181] gatherer "clusterconfig" function "metrics" took 3.717805ms to process 0 records I0705 02:16:14.641496 1 tasks_processing.go:74] worker 6 stopped. E0705 02:16:14.643673 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.144.49:9094: connect: connection refused I0705 02:16:14.643700 1 tasks_processing.go:74] worker 28 stopped. E0705 02:16:14.643707 1 gather.go:144] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.144.49:9094: connect: connection refused I0705 02:16:14.643721 1 gather.go:181] gatherer "clusterconfig" function "active_alerts" took 2.387519ms to process 0 records I0705 02:16:14.645570 1 tasks_processing.go:74] worker 11 stopped. E0705 02:16:14.645590 1 gather.go:144] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0705 02:16:14.645601 1 gather.go:181] gatherer "clusterconfig" function "support_secret" took 28.669545ms to process 0 records I0705 02:16:14.645682 1 tasks_processing.go:74] worker 27 stopped. I0705 02:16:14.645827 1 recorder.go:70] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=41458520cb084bb8fed43c471a8a501296ab936c6d22af737642eb90495b473d I0705 02:16:14.646036 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=98a9331bc406cbc9f0190f411669922219882ef15a4fff9619694b5a7bd0d677 I0705 02:16:14.646246 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=a96c39474d718d2045393e5c52f65c2011125c47031490522a825e3fa2b136c3 I0705 02:16:14.646327 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=38926f2e6ff92ba9d68522a7d763e5b327a7f67d09fcadad3c1e8c308e3355e8 I0705 02:16:14.646385 1 recorder.go:70] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=d09d911b281ae73cddea726ddcc47eda7ad7805c390514196db78fa2c2b21010 I0705 02:16:14.646451 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=78527dc1a680e748353744d0c509f4ac7517d9c4f59a8126ec573c481cf0e7f5 I0705 02:16:14.646510 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=5302f184d2f60530cb67dfa9330099fc388fa6cec938b2eaaf5220fcfd6c8fb4 I0705 02:16:14.646574 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=daf336c9b7e666afc4ee9fb8791b2945917e92ba322c29715ffd2591a8c173d9 I0705 02:16:14.646632 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=b1813d5d73f38466aa09fc5bc03db9b15a497437aa25598bb1717f5014ef3e33 I0705 02:16:14.646692 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=7c0f411779f39723e7e4681c8db7a1b959d27200dec0f3c74e2e3c19e1124b28 I0705 02:16:14.646731 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=d8cef64a33eefc4860ea02c5c4deb231c041422fbbbb654188e5fd942e44c327 I0705 02:16:14.646770 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=2c16a49a3de976a12a1e5b472bc94f5465523f8215f7fb49441a4f86a6ab0238 I0705 02:16:14.646808 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=82e34cbce5f939a465c201e49180e2f9a0fa630b0eb6245fdeeaf60677bf0c36 I0705 02:16:14.646821 1 gather.go:181] gatherer "clusterconfig" function "validating_webhook_configurations" took 24.72985ms to process 13 records I0705 02:16:14.646896 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0705 02:16:14.646910 1 gather.go:181] gatherer "clusterconfig" function "overlapping_namespace_uids" took 38.484651ms to process 1 records I0705 02:16:14.646918 1 gather.go:181] gatherer "clusterconfig" function "jaegers" took 10.412426ms to process 0 records I0705 02:16:14.646925 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 10.707526ms to process 0 records I0705 02:16:14.646931 1 gather.go:181] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 25.194255ms to process 0 records I0705 02:16:14.646939 1 gather.go:181] gatherer "clusterconfig" function "machine_sets" took 10.667245ms to process 0 records I0705 02:16:14.646948 1 tasks_processing.go:74] worker 24 stopped. I0705 02:16:14.646954 1 tasks_processing.go:74] worker 15 stopped. I0705 02:16:14.646960 1 tasks_processing.go:74] worker 9 stopped. I0705 02:16:14.646964 1 tasks_processing.go:74] worker 0 stopped. I0705 02:16:14.646969 1 tasks_processing.go:74] worker 22 stopped. I0705 02:16:14.647774 1 tasks_processing.go:74] worker 25 stopped. I0705 02:16:14.647794 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0705 02:16:14.647806 1 gather.go:181] gatherer "clusterconfig" function "dvo_metrics" took 39.673101ms to process 1 records I0705 02:16:14.648373 1 tasks_processing.go:74] worker 3 stopped. I0705 02:16:14.648584 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=5eea5d5f355e973de54305daa2fa93bb64e65b7805993b4ff33253131096dce0 I0705 02:16:14.648691 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=a04a09f04e91ce630135bcd6e9119470cee9455fb7f0dd3c09ffca23283fdcd7 I0705 02:16:14.648717 1 gather.go:181] gatherer "clusterconfig" function "clusterroles" took 41.287453ms to process 2 records I0705 02:16:14.658844 1 tasks_processing.go:74] worker 23 stopped. I0705 02:16:14.658858 1 gather.go:181] gatherer "clusterconfig" function "storage_cluster" took 21.342855ms to process 0 records I0705 02:16:14.666184 1 tasks_processing.go:74] worker 26 stopped. I0705 02:16:14.666222 1 gather.go:181] gatherer "clusterconfig" function "openstack_controlplanes" took 29.248415ms to process 0 records I0705 02:16:14.666428 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0705 02:16:14.666442 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0705 02:16:14.666446 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0705 02:16:14.666452 1 controller.go:203] Source scaController *sca.Controller is not ready I0705 02:16:14.666456 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0705 02:16:14.666487 1 controller.go:457] The operator is still being initialized I0705 02:16:14.666506 1 controller.go:482] The operator is healthy I0705 02:16:14.668757 1 tasks_processing.go:74] worker 5 stopped. I0705 02:16:14.668777 1 gather.go:181] gatherer "clusterconfig" function "certificate_signing_requests" took 31.613821ms to process 0 records I0705 02:16:14.669343 1 tasks_processing.go:74] worker 8 stopped. I0705 02:16:14.669427 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0705 02:16:14.669532 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=8880b7cffb68a6e523d87ad9cef5bf607cedbda2890d27ae4e773f51672e562f I0705 02:16:14.669546 1 gather.go:181] gatherer "clusterconfig" function "schedulers" took 31.15861ms to process 1 records I0705 02:16:14.669633 1 tasks_processing.go:74] worker 29 stopped. I0705 02:16:14.669715 1 recorder.go:70] Recording config/featuregate with fingerprint=d35c7dac06e33c2104cc9aedccc38386bd51ee0f144408802146ac25263a3b0d I0705 02:16:14.669727 1 gather.go:181] gatherer "clusterconfig" function "feature_gates" took 32.513778ms to process 1 records I0705 02:16:14.670194 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0705 02:16:14.670384 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0705 02:16:14.670468 1 tasks_processing.go:74] worker 17 stopped. I0705 02:16:14.674603 1 base_controller.go:73] Caches are synced for ConfigController I0705 02:16:14.674626 1 base_controller.go:110] Starting #1 worker of ConfigController controller ... I0705 02:16:14.674730 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-wdc2t with fingerprint=192a26ea866fa91cda42ab1178f76dca1fa229981185881e835dda76c9f6a68a I0705 02:16:14.675557 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-2xgtc with fingerprint=1d6eb67beed3f35090f5d24dbb7745f42af37484e1273e04f7e156025ccc7565 I0705 02:16:14.675680 1 recorder.go:70] Recording config/running_containers with fingerprint=4598db988101e8b35f1c588584bba31e7141c36a4f698a32521b7ae08831a49e I0705 02:16:14.675694 1 gather.go:181] gatherer "clusterconfig" function "container_images" took 49.407427ms to process 3 records I0705 02:16:14.675793 1 tasks_processing.go:74] worker 1 stopped. I0705 02:16:14.675904 1 recorder.go:70] Recording config/ingress with fingerprint=6d86985d0864ca4ca7fcc120e5dbfcbe36f459eb93910bc60c38486b81cee11e I0705 02:16:14.675921 1 gather.go:181] gatherer "clusterconfig" function "ingress" took 37.706326ms to process 1 records I0705 02:16:14.675986 1 recorder.go:70] Recording config/olm_operators with fingerprint=23ba67b59ea3885134ce676b77d177c9a414d3d114e7d83051a453171e4c66b4 I0705 02:16:14.675993 1 tasks_processing.go:74] worker 18 stopped. I0705 02:16:14.675997 1 gather.go:181] gatherer "clusterconfig" function "olm_operators" took 54.464618ms to process 1 records E0705 02:16:14.676007 1 gather.go:144] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0705 02:16:14.676032 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0705 02:16:14.676047 1 gather.go:181] gatherer "clusterconfig" function "machine_configs" took 36.068266ms to process 1 records I0705 02:16:14.676056 1 tasks_processing.go:74] worker 19 stopped. I0705 02:16:14.676147 1 requests.go:204] Asking for SCA certificate for x86_64 architecture I0705 02:16:14.680728 1 tasks_processing.go:74] worker 4 stopped. I0705 02:16:14.681379 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=4b4158603edb7fccb42a8c53871378387b0907b8edd6222945f7e8d4155652e8 I0705 02:16:14.681605 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=dcf30974d31652980d1e4ac0270cf5b4c844043a40ccabe6a30f74d76e6984b0 I0705 02:16:14.681618 1 gather.go:181] gatherer "clusterconfig" function "crds" took 44.928607ms to process 2 records I0705 02:16:14.681629 1 gather.go:181] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 64.167953ms to process 0 records I0705 02:16:14.681647 1 tasks_processing.go:74] worker 7 stopped. I0705 02:16:14.681712 1 tasks_processing.go:74] worker 16 stopped. I0705 02:16:14.681994 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=7b01638f4895106641b561c8756be04d16bf0acf3b826e1e6e5063cee062ea2a I0705 02:16:14.682007 1 gather.go:181] gatherer "clusterconfig" function "image_registries" took 44.786848ms to process 1 records I0705 02:16:14.686140 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:16:14.701620 1 tasks_processing.go:74] worker 10 stopped. E0705 02:16:14.701639 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0705 02:16:14.701646 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0705 02:16:14.701650 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0705 02:16:14.701687 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0705 02:16:14.701702 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0705 02:16:14.701708 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=e1a0ad594db3a5d5d50e9e25d7b235af20f84f5def41537f127febe525b834f2 I0705 02:16:14.701717 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0705 02:16:14.701755 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0705 02:16:14.701771 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0705 02:16:14.701785 1 gather.go:181] gatherer "clusterconfig" function "config_maps" took 94.086342ms to process 6 records I0705 02:16:14.728727 1 tasks_processing.go:74] worker 2 stopped. I0705 02:16:14.729309 1 recorder.go:70] Recording config/node/ip-10-0-0-109.ec2.internal with fingerprint=b76425ff33c49648fa844716d5b0b7cc9711d28ef1739decd30335edf0244e87 I0705 02:16:14.729412 1 recorder.go:70] Recording config/node/ip-10-0-1-56.ec2.internal with fingerprint=d1ca0c4168810761358dd0ca9a011a072e55e3ce915d6385edf628a15d6d89a8 I0705 02:16:14.729508 1 recorder.go:70] Recording config/node/ip-10-0-2-134.ec2.internal with fingerprint=8966c4957d2ae13c8f1530779f54abc436ab6a4cbecf2a0d8edcf4ead2538157 I0705 02:16:14.729525 1 gather.go:181] gatherer "clusterconfig" function "nodes" took 91.777783ms to process 3 records I0705 02:16:14.737235 1 base_controller.go:73] Caches are synced for LoggingSyncer I0705 02:16:14.737251 1 base_controller.go:110] Starting #1 worker of LoggingSyncer controller ... I0705 02:16:14.737908 1 tasks_processing.go:74] worker 14 stopped. I0705 02:16:14.738623 1 recorder.go:70] Recording config/version with fingerprint=18378974c50730018f80c729223bc86b6609c1c993803397190492a051b21ae6 I0705 02:16:14.738646 1 recorder.go:70] Recording config/id with fingerprint=04b19eefd93aba3744d21af24300077662e4c6f2e8e76aa67470340fbc042f5c I0705 02:16:14.738653 1 gather.go:181] gatherer "clusterconfig" function "version" took 128.963402ms to process 2 records I0705 02:16:14.767080 1 tasks_processing.go:74] worker 31 stopped. I0705 02:16:14.767097 1 gather.go:181] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 129.297802ms to process 0 records I0705 02:16:14.769859 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress-operator" I0705 02:16:14.780073 1 tasks_processing.go:74] worker 21 stopped. E0705 02:16:14.780093 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0705 02:16:14.780100 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rbmo3u9i3bmqb43bcejvjtgghhlk02v-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rbmo3u9i3bmqb43bcejvjtgghhlk02v-primary-cert-bundle-secret" not found I0705 02:16:14.780169 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=e352bf5a33e2f8f941f14a427d98907612d9f790e6a4fda08600b180301fc377 I0705 02:16:14.780193 1 gather.go:181] gatherer "clusterconfig" function "ingress_certificates" took 171.787081ms to process 1 records I0705 02:16:14.780340 1 cluster_transfer.go:102] no available accepted cluster transfer I0705 02:16:14.780352 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0705 02:16:14.808851 1 gather_cluster_operator_pods_and_events.go:121] Found 5 pods with 5 containers I0705 02:16:14.808866 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 5033164 bytes I0705 02:16:14.809074 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5777b9b74c-nhh6d pod in namespace openshift-ingress (previous: false). I0705 02:16:14.864969 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6478677768-5trtk pod in namespace openshift-ingress (previous: false). I0705 02:16:14.904069 1 sca.go:141] etc-pki-entitlement secret successfully updated I0705 02:16:14.904088 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated in the etc-pki-entitlement secret I0705 02:16:14.953092 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6478677768-k8qbr pod in namespace openshift-ingress (previous: false). I0705 02:16:14.974411 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-4hjrb pod in namespace openshift-ingress-canary (previous: false). I0705 02:16:14.996790 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-7rw98 pod in namespace openshift-ingress-canary (previous: false). I0705 02:16:15.021080 1 tasks_processing.go:74] worker 20 stopped. I0705 02:16:15.021270 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=430b5b443a773468817b20a051e0055609d9343ee94e0bbb607812251bf48aa2 I0705 02:16:15.021357 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=b0c621629f8a4a03f1220a825d18bed7e97ea73a5dd286a4caeb53e2378388ad I0705 02:16:15.021393 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=fae13abf363532fe1a2e70ec66c981c14d2aa035dbeacd85d25f61035a08abde I0705 02:16:15.021414 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-5777b9b74c-nhh6d/router_current.log with fingerprint=7ad5ec4658393ebbf523b2b8c8f16c22a2e935235ece13d29fcf03be2042e8c0 I0705 02:16:15.021428 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-6478677768-5trtk/router_current.log with fingerprint=8e18ace05f12ad06ddd2a137a1446ab63839f739a6dc1b4ec0a6d8c36b226db9 I0705 02:16:15.021442 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-6478677768-k8qbr/router_current.log with fingerprint=2ba86940ef265c7be351966a8c73a68951522bfaef8d50200470e868923eebb5 I0705 02:16:15.021449 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-4hjrb/serve-healthcheck-canary_current.log with fingerprint=bc69c045ecc5fb8a72c945f86c4adaae63cbe15093a6919afebd162a6971ddf2 I0705 02:16:15.021458 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-7rw98/serve-healthcheck-canary_current.log with fingerprint=121b1e60fe64f8af1699583d705a024aabbd3e2b7b225b7b96630f26d9341b67 I0705 02:16:15.021471 1 gather.go:181] gatherer "clusterconfig" function "operators_pods_and_events" took 380.534255ms to process 8 records I0705 02:16:15.287033 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0705 02:16:15.486564 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0705 02:16:16.890496 1 tasks_processing.go:74] worker 12 stopped. I0705 02:16:16.890657 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=6f45b1314350c10884602e363c6f5cf6cff570a736cc2678604d608d8f2bc4d3 I0705 02:16:16.890705 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=2dadbbc090754506ccdfd76e54ddbded587c0c5c3e76dd1419a6e56b93f627c3 I0705 02:16:16.890725 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=84caf2187f844c0c33c39fd4d2695f1c33926336c96468fc504355eddada99a7 I0705 02:16:16.890734 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0705 02:16:16.890756 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=f9e68312d34c472e0f43d9cac9aeff1a7161ff17af97ef03d05931e948d28552 I0705 02:16:16.890775 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0705 02:16:16.890796 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=3b3871760bb776096c161ec1f5fdb9687333cb15cb84fab9b6f2e61acb1a1df0 I0705 02:16:16.890823 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=56b54a96b2d73e437b90ce25e6e9430dedd050733a730317ff86567e7c747681 I0705 02:16:16.890851 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=6bc92fa848de4c477d3ed09be1f6c8fc1ac77e5ac30ae77afeaed4df826a9a47 I0705 02:16:16.890882 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=010f0db0c431c260bfec607ffba1571cb8d6d11fe0485748d595c4ea97f60299 I0705 02:16:16.891169 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=1024a52fc23586320bc1c1f26ec60c4e5df1f7b23f1e8ee97df4590613904b2a I0705 02:16:16.891188 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=9c6c94f9b07f7457fbb6b78a3b900e409052bd930d690f4bcf5ff0fe91bb9861 I0705 02:16:16.891197 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0705 02:16:16.891234 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=3e0638485523bf8acc15f9c3cc3a727e5a07c0d08fc661adf67a475af0040916 I0705 02:16:16.891247 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0705 02:16:16.891291 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=af755e3b9c66c273311487a98c129e876ef43d2254e043411d39018f9720e80a I0705 02:16:16.891301 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0705 02:16:16.891322 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=5211dcb08338e8a614e8f8b4ee68fc619acb1fe90766892d2452270da30850d1 I0705 02:16:16.891332 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0705 02:16:16.891353 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=9a9037d81c6a23a047388314ad11b78adfe453f7c8cde8be837f3a8c53022aa9 I0705 02:16:16.891457 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=86765ec65daea9f5388ef22206c4693c7e4d0e9d91e817e865ac8b8b61d9c9a6 I0705 02:16:16.891481 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0705 02:16:16.891501 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=3831fb7a5fd6532b7a0a2f6fb7d8a26becb9617d2e808329d7e930a2be3fbebc I0705 02:16:16.891524 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=87623a9078e036e2c5d8b0e63960704256394dd1ca823004083d159e1bcef0ee I0705 02:16:16.891533 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0705 02:16:16.891548 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=433402c0a14398b3191396ea43bf23ce22a136ff7b76302036c8c61e1c144264 I0705 02:16:16.891556 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0705 02:16:16.891572 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=4a245874534f183ce79c25386f0a67fe287397764371556f711e439819b561ed I0705 02:16:16.891581 1 recorder.go:70] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0705 02:16:16.891594 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=e9399f81d27da0b74d66fd359b174614746bdada45ace2afac15527d56ea0f3d I0705 02:16:16.891610 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=40ef0dec10bed3ffadf952fbf89cbad4fa59586de94ffcfe6ef84a299fee0bfb I0705 02:16:16.891625 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=323f8253bd384b178eb1ff43db22e99776a3cf77df20c2a5177b0584158e1d25 I0705 02:16:16.891645 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=1b3f98d4eadf90051432ecc24ca39932a611f326839b8b23d0007bc093a9fa76 I0705 02:16:16.891655 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0705 02:16:16.891678 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=0417bc73600b544224a036613d56a9f2a544b514e42b767a528b827c5969ea0a I0705 02:16:16.891695 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0705 02:16:16.891704 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0705 02:16:16.891720 1 gather.go:181] gatherer "clusterconfig" function "operators" took 2.283397061s to process 37 records I0705 02:16:27.653417 1 tasks_processing.go:74] worker 30 stopped. I0705 02:16:27.653455 1 recorder.go:70] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0705 02:16:27.653471 1 gather.go:181] gatherer "clusterconfig" function "install_plans" took 13.046322441s to process 1 records I0705 02:16:28.443785 1 tasks_processing.go:74] worker 13 stopped. I0705 02:16:28.444134 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=ba16ea6b4503917d33437d4d1999039313b5aa3c669ab94c714d5681918865aa I0705 02:16:28.444163 1 gather.go:181] gatherer "clusterconfig" function "service_accounts" took 13.806049455s to process 1 records E0705 02:16:28.444259 1 periodic.go:252] clusterconfig failed after 13.837s with: function "tsdb_status" failed with an error, function "silenced_alerts" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error I0705 02:16:28.444277 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "tsdb_status" failed with an error, function "silenced_alerts" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error I0705 02:16:28.444287 1 periodic.go:214] Running workloads gatherer I0705 02:16:28.444308 1 tasks_processing.go:45] number of workers: 2 I0705 02:16:28.444325 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 02:16:28.444331 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0705 02:16:28.444338 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 02:16:28.444353 1 tasks_processing.go:71] worker 0 working on workload_info task. I0705 02:16:28.473730 1 gather_workloads_info.go:281] Loaded pods in 0s, will wait 24s for image data I0705 02:16:28.476610 1 tasks_processing.go:74] worker 1 stopped. I0705 02:16:28.476631 1 gather.go:181] gatherer "workloads" function "helmchart_info" took 32.265083ms to process 0 records I0705 02:16:28.481481 1 gather_workloads_info.go:390] No image sha256:0c1abef76d6d2832ddcff31e950ffafeb4e0f6fad80a5f5dca94591de0fc9581 (9ms) I0705 02:16:28.489792 1 gather_workloads_info.go:390] No image sha256:db73c8cfc081ee27ef2f13fb2a1e004307b174e60c1af45121c0284d1f2af2bb (8ms) I0705 02:16:28.498240 1 gather_workloads_info.go:390] No image sha256:5bcfb1b739fa4e6ff9e21e4843f8520f29608135341f7d8dcc31084b9ba25f1d (8ms) I0705 02:16:28.507112 1 gather_workloads_info.go:390] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (9ms) I0705 02:16:28.515383 1 gather_workloads_info.go:390] No image sha256:cf34884c4154c8b470328a8f1fd45ad5f250b8899c57b897bfa0af41fa6efe18 (8ms) I0705 02:16:28.523696 1 gather_workloads_info.go:390] No image sha256:13aabbaf260d634ff0e9ac1af88af075b5ba671fd51313e33e74d7d2f7e54128 (8ms) I0705 02:16:28.531251 1 gather_workloads_info.go:390] No image sha256:726adbde6a230e7b4da4a27508a67875b5579d80f6cdeb3324f90993d4de386a (8ms) I0705 02:16:28.538995 1 gather_workloads_info.go:390] No image sha256:50076d29400ecf9b704d8175509fc8cb7c28bea91d46a28dc37cbda1668c69f9 (8ms) I0705 02:16:28.546748 1 gather_workloads_info.go:390] No image sha256:ed0c9631d6e424912cd8ce78ca7a2c4f76b3fc15e9e34fe00999170ccf19a611 (8ms) I0705 02:16:28.554439 1 gather_workloads_info.go:390] No image sha256:1eee856bde9bad71c4444c494225a0cf02d1f4a5293acf0bd6e7e1814efa0064 (8ms) I0705 02:16:28.580531 1 gather_workloads_info.go:390] No image sha256:0158b297aec0a66376c04d3c7cf4828db5e5d9f424e331ff7c7b796828036875 (26ms) I0705 02:16:28.681885 1 gather_workloads_info.go:390] No image sha256:351ca92471647f383f3cd7f0b70f33f5fa11c39ab569c4948dff7d471a6c8be3 (101ms) I0705 02:16:28.782111 1 gather_workloads_info.go:390] No image sha256:02283a64c1936a5dd09d5c6bc0ecbcf9196fe4df6a4055c8be22cebd850aa7a3 (100ms) I0705 02:16:28.883163 1 gather_workloads_info.go:390] No image sha256:612ffcbc92e84af62541d34ca47957fe0a683fe504ec87d1998cacbd8180f793 (101ms) I0705 02:16:28.983284 1 gather_workloads_info.go:390] No image sha256:70f4702845ec5b627d49504a53a13d997c0578630cddffb89aba5cfebe04fc25 (100ms) I0705 02:16:29.081778 1 gather_workloads_info.go:390] No image sha256:3fea7db586d3236c1876a8b523af0d8d65c84d6f6cf5d0f4e41cafcffe6db5b5 (98ms) I0705 02:16:29.182202 1 gather_workloads_info.go:390] No image sha256:5e87ba403f9e9fff3002a8275aba0d6c36aed54fb4754d90fa0516ad989f187a (100ms) I0705 02:16:29.283249 1 gather_workloads_info.go:401] Found image sha256:cdba3ca6e2b89d411b5ef466468bc8d91a07bf4bd14238baa65fb4397f54257d (101ms) I0705 02:16:29.382826 1 gather_workloads_info.go:390] No image sha256:8f34d18e3a237f96e2b9e9a071e5bf159c04f978ed12af457f1d065e6796220d (99ms) I0705 02:16:29.481943 1 gather_workloads_info.go:390] No image sha256:194ee1155b87e110bc6c4a6e55667e41f3bdc8292e26a9561f9082fdfe467eec (99ms) I0705 02:16:29.582541 1 gather_workloads_info.go:390] No image sha256:42b4c6bdf3081ec264bc5edbc4eb779151de9e4742d6fe59a316645891df649c (101ms) I0705 02:16:29.681966 1 gather_workloads_info.go:390] No image sha256:06821dfe231330e0eb5be915b9468cd13826a7ee59d1a732b89cb140548eade2 (99ms) I0705 02:16:29.784438 1 gather_workloads_info.go:390] No image sha256:f132095bb816921d7780c4525219aa183a9fc18975e0e615c98374c2bc5a9d3b (102ms) I0705 02:16:29.884311 1 gather_workloads_info.go:390] No image sha256:586127c49e25052113263d8aeaf6a1623aef22987364f119a17b2848982ff716 (100ms) I0705 02:16:29.993042 1 gather_workloads_info.go:390] No image sha256:24519dcc0e63cdc60a639dd474d06e7612e9b76966bc60322cd9026a0f1b878c (109ms) I0705 02:16:30.083756 1 gather_workloads_info.go:390] No image sha256:a7fc4d05c4a3c82c9f355ca985d6fd26033c0d6a98716f25fcd2f54bae1d2fa0 (91ms) I0705 02:16:30.182286 1 gather_workloads_info.go:390] No image sha256:221622a101db9797df0622b4534e3e42c4498df458f355a8a02fad8d65f8e3f2 (98ms) I0705 02:16:30.282367 1 gather_workloads_info.go:390] No image sha256:45b1f11898d403139a45977726e024816081b684d6f9cb4eb9093fb2881f40dd (100ms) I0705 02:16:30.382358 1 gather_workloads_info.go:390] No image sha256:8e158c83ec12b3e0237fab9a221630bd691e79c079a111f346354ff43b8199df (100ms) I0705 02:16:30.482530 1 gather_workloads_info.go:390] No image sha256:3969660e807e8ce5656a74e5816573656241946ef1d766a9d2e21d434356313c (100ms) I0705 02:16:30.581477 1 gather_workloads_info.go:390] No image sha256:4dcdadc14b7bdb2c00954d92e957d63aafbbe5f4047fe028e575d65725923cb0 (99ms) I0705 02:16:30.682075 1 gather_workloads_info.go:390] No image sha256:3a2c82b772ea6072e1eca0efc5303f002189e80b0e985fb3e3b8ea9a12fe8060 (101ms) I0705 02:16:30.783922 1 gather_workloads_info.go:390] No image sha256:48dff3fcb0da58eb9b2b500f89af8b7d0790ffa60da0b0e301feca971dbafcbb (102ms) I0705 02:16:30.882005 1 gather_workloads_info.go:390] No image sha256:2897724cec17bc13acc133181f64ccb8d54ce5dc9582f9cecf704c2540e15600 (98ms) I0705 02:16:30.983586 1 gather_workloads_info.go:390] No image sha256:967542857e0607674105c25151e0470aa8f8427f4b7b4598b5933ce80e79887f (102ms) I0705 02:16:31.083767 1 gather_workloads_info.go:390] No image sha256:c96b4bddf3dd7daaa797cbda4dd92e981de2f5fc3404c01381f31dc0414d5c8d (100ms) I0705 02:16:31.183288 1 gather_workloads_info.go:401] Found image sha256:b8781637560d49de9b8e8cdd59ec75f03d98921ea66950381849398128cc9cb9 (99ms) I0705 02:16:31.282151 1 gather_workloads_info.go:390] No image sha256:4b9b7949d209e257a76053de4ac74193f747c0d42e25dcd270d655849a4e51aa (99ms) I0705 02:16:31.382012 1 gather_workloads_info.go:390] No image sha256:8562a483f4568254250f19fc68d79f88c94a48508aefe59a70c1021a3aad000a (100ms) I0705 02:16:31.482004 1 gather_workloads_info.go:390] No image sha256:c7aa910ca06f8fd89cdf2519136e0f3533bdacfc6bd58a3f6683f9f08756c7b1 (100ms) I0705 02:16:31.582576 1 gather_workloads_info.go:401] Found image sha256:751bd7eb62f632c0eef0ca91b6c57c64d4f1279d4dbe1a9b875ac4020a7b693f (101ms) I0705 02:16:31.681781 1 gather_workloads_info.go:390] No image sha256:30ff5e5b72f274598704ca69f1832eba1040cc25da47386004fd59500b6e4c55 (99ms) I0705 02:16:31.782069 1 gather_workloads_info.go:390] No image sha256:0c6b7f828184b16a5e6630111e0715e11d34dae9b0bf64619466543e6168e05c (100ms) I0705 02:16:31.882003 1 gather_workloads_info.go:390] No image sha256:6a9b4fd887fa7f1729302fb0330c3c9a483197bf78becdcfda4235ec823a2c39 (100ms) I0705 02:16:31.984907 1 gather_workloads_info.go:390] No image sha256:894452e6587de55d04e37163d3da53e11fced8fb08bd0acf61e6d49f7964284e (103ms) I0705 02:16:32.083055 1 gather_workloads_info.go:390] No image sha256:09b3e9e6a4f382fb4bbdb67bc945bcf19d66f604b7c8ffd66d4d4a2c94224248 (98ms) I0705 02:16:32.183127 1 gather_workloads_info.go:390] No image sha256:0242cefbf0bb876101c4d67ab4e07401cb0ba6c02be2b73a4f17ba33b2960005 (100ms) I0705 02:16:32.282263 1 gather_workloads_info.go:390] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (99ms) I0705 02:16:32.387552 1 gather_workloads_info.go:390] No image sha256:e67b1ecadffe8bcaa8953d449d1559f5419c3296f6fc065f34d53985b07becdb (105ms) I0705 02:16:32.481571 1 gather_workloads_info.go:390] No image sha256:31a22f6ee061d0f6558664e580ec0f80f1927d4ec29626cf3e9420837f5d80e5 (94ms) I0705 02:16:32.581760 1 gather_workloads_info.go:390] No image sha256:80fdd5049bd2815e49c198a795a116976a852e8de630be74872439ba6c08758a (100ms) I0705 02:16:32.685072 1 gather_workloads_info.go:390] No image sha256:41029e57ece558b2e0b12174ce924647fd9d5c5d83aadf4bba03d6916d3f9f54 (103ms) I0705 02:16:32.788635 1 gather_workloads_info.go:390] No image sha256:c8ba3078651d566566f7855c51c53baedfe6b6c743d316275e81b6e9d242bfb6 (104ms) I0705 02:16:32.887631 1 gather_workloads_info.go:390] No image sha256:38787de16ee8c686305f93b501d891b50cde24136009863869d230c0d4eeee5c (99ms) I0705 02:16:32.985239 1 gather_workloads_info.go:390] No image sha256:788d16bfff4bd6bb3ce97acc643b56fae6e619674ab279535480e519b4d5b769 (98ms) I0705 02:16:33.081400 1 gather_workloads_info.go:390] No image sha256:68abf16a3b6c33fdd22a823c7d00f96aaaa8015d370116962a5835984908645a (96ms) I0705 02:16:33.181779 1 gather_workloads_info.go:390] No image sha256:1c7c24111f1709d7648156873ced09462638aa4b3cf7e7aee13297e51bcdfb3e (100ms) I0705 02:16:33.281957 1 gather_workloads_info.go:390] No image sha256:bc829520eeec090b60d17abbad1a22b34ea99b426193395808a778d73a5fb57a (100ms) I0705 02:16:33.384084 1 gather_workloads_info.go:390] No image sha256:6c12532a36e461e69227c6d5266dce4506e963567ec689a8cf6fd2e70ae20bf4 (102ms) I0705 02:16:33.384119 1 tasks_processing.go:74] worker 0 stopped. I0705 02:16:33.384480 1 recorder.go:70] Recording config/workload_info with fingerprint=3d0b69d1f7012b58b36096dcf38bda25702e0af576da6f0d8df677e870648aa2 I0705 02:16:33.384498 1 gather.go:181] gatherer "workloads" function "workload_info" took 4.939759051s to process 1 records I0705 02:16:33.384511 1 periodic.go:261] Periodic gather workloads completed in 4.94s I0705 02:16:33.384519 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0705 02:16:33.384523 1 periodic.go:214] Running conditional gatherer I0705 02:16:33.395938 1 requests.go:282] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.18.9/gathering_rules I0705 02:16:33.526945 1 conditional_gatherer.go:336] updating alerts cache for conditional gatherer E0705 02:16:33.529277 1 conditional_gatherer.go:322] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.40.21:9091: connect: connection refused I0705 02:16:33.529357 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0705 02:16:33.542647 1 conditional_gatherer.go:390] cluster version is '4.18.9' E0705 02:16:33.542666 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542675 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542681 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542685 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542688 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542691 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542694 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542698 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542701 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0705 02:16:33.542704 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0705 02:16:33.542722 1 tasks_processing.go:45] number of workers: 3 I0705 02:16:33.542737 1 tasks_processing.go:69] worker 2 listening for tasks. I0705 02:16:33.542744 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0705 02:16:33.542756 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 02:16:33.542762 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 02:16:33.542834 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0705 02:16:33.542828 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0705 02:16:33.542848 1 tasks_processing.go:74] worker 0 stopped. I0705 02:16:33.542920 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0705 02:16:33.542942 1 gather.go:181] gatherer "conditional" function "remote_configuration" took 1.348µs to process 1 records I0705 02:16:33.543018 1 tasks_processing.go:74] worker 1 stopped. I0705 02:16:33.543032 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0705 02:16:33.543041 1 gather.go:181] gatherer "conditional" function "conditional_gatherer_rules" took 2.134µs to process 1 records I0705 02:16:33.543052 1 gather.go:181] gatherer "conditional" function "rapid_container_logs" took 205.952µs to process 0 records I0705 02:16:33.543058 1 tasks_processing.go:74] worker 2 stopped. I0705 02:16:33.543076 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=AsExpected message= I0705 02:16:33.543093 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f W0705 02:16:33.579127 1 gather.go:213] can't read cgroups memory usage data: open /sys/fs/cgroup/memory/memory.usage_in_bytes: no such file or directory I0705 02:16:33.579310 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=9fdba06c1e250ef005b8ed2d4a2cb885182da7edad3576b959a48804ef77aa40 I0705 02:16:33.579438 1 diskrecorder.go:70] Writing 111 records to /var/lib/insights-operator/insights-2026-07-05-021633.tar.gz I0705 02:16:33.587244 1 diskrecorder.go:51] Wrote 111 records to disk in 7ms I0705 02:16:33.587279 1 periodic.go:283] Gathering cluster info every 2h0m0s I0705 02:16:33.587295 1 periodic.go:284] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0705 02:17:44.607716 1 diskrecorder.go:170] Found files to send: insights-2026-07-05-021633.tar.gz I0705 02:17:44.607746 1 insightsuploader.go:150] Checking archives to upload periodically every 16m29.468697729s I0705 02:17:44.607755 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0705 02:17:44.622333 1 requests.go:47] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0705 02:17:44.755656 1 requests.go:88] Successfully reported id=2026-07-05T02:17:44Z x-rh-insights-request-id=45f54ba2b3e34371907cb626f7fdb6bd, wrote=58198 I0705 02:17:44.755777 1 insightsuploader.go:187] Uploaded report successfully in 148.010767ms I0705 02:17:44.755802 1 controller.go:119] Initializing last reported time to 2026-07-05T02:17:44Z I0705 02:17:44.755822 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0705 02:17:44.755835 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0705 02:17:44.755844 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0705 02:17:44.762032 1 controller.go:482] The operator is healthy I0705 02:18:14.644331 1 controller.go:482] The operator is healthy I0705 02:18:49.549391 1 insightsreport.go:137] Pulling report from smart-proxy I0705 02:18:49.549417 1 insightsreport.go:149] Retrieving report I0705 02:18:49.559425 1 requests.go:111] Retrieving report for cluster: 420d48b0-abe8-48e8-a615-9a5aff43063a I0705 02:18:49.559439 1 requests.go:112] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/420d48b0-abe8-48e8-a615-9a5aff43063a/reports I0705 02:18:49.562301 1 requests.go:122] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/420d48b0-abe8-48e8-a615-9a5aff43063a/reports I0705 02:18:49.740802 1 insightsreport.go:184] Report retrieved I0705 02:18:49.790623 1 insightsreport.go:239] Report retrieved correctly I0705 02:20:14.645249 1 controller.go:482] The operator is healthy I0705 02:21:14.600389 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0705 02:21:14.606572 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:21:14.606591 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0705 02:21:14.611038 1 secretconfigobserver.go:119] support secret does not exist I0705 02:22:14.670733 1 controller.go:482] The operator is healthy I0705 02:24:14.647373 1 controller.go:482] The operator is healthy I0705 02:26:14.594003 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:26:14.612063 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0705 02:26:14.617732 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:26:14.617753 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0705 02:26:14.622599 1 secretconfigobserver.go:119] support secret does not exist I0705 02:26:14.642629 1 controller.go:482] The operator is healthy I0705 02:28:14.646585 1 controller.go:482] The operator is healthy I0705 02:30:14.642656 1 controller.go:482] The operator is healthy I0705 02:31:14.623080 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0705 02:31:14.630139 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:31:14.630171 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0705 02:31:14.635202 1 secretconfigobserver.go:119] support secret does not exist I0705 02:32:14.645683 1 controller.go:482] The operator is healthy I0705 02:34:14.702360 1 controller.go:482] The operator is healthy I0705 02:34:19.226435 1 insightsuploader.go:146] Nothing to report since 2026-07-05T02:17:44Z I0705 02:36:14.591747 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:36:14.635986 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0705 02:36:14.641471 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:36:14.641496 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0705 02:36:14.643445 1 controller.go:482] The operator is healthy I0705 02:36:14.646276 1 secretconfigobserver.go:119] support secret does not exist I0705 02:38:14.643840 1 controller.go:482] The operator is healthy