I0704 22:24:16.485471 1 cmd.go:250] Using service-serving-cert provided certificates I0704 22:24:16.485904 1 observer_polling.go:159] Starting file observer W0704 22:24:16.507257 1 builder.go:266] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-6fd96f47fd" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0704 22:24:17.268079 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0704 22:24:17.268302 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0704 22:24:17.269637 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0704 22:24:17.269649 1 secure_serving.go:57] Forcing use of http/1.1 only W0704 22:24:17.269749 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0704 22:24:17.269757 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0704 22:24:17.269770 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0704 22:24:17.269775 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0704 22:24:17.269786 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0704 22:24:17.269790 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0704 22:24:17.272898 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes ChunkSizeMiB CloudDualStackNodeIPs DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy PersistentIPsForVirtualization PrivateHostedZoneAWS SetEIPForNLBIngressController VSphereControlPlaneMachineSet VSphereDriverConfiguration VSphereMultiVCenters VSphereStaticIPs ValidatingAdmissionPolicy AWSClusterHostedDNS AdditionalRoutingCapabilities AutomatedEtcdBackup BootcNodeManagement CSIDriverSharedResource ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ConsolePluginContentSecurityPolicy DNSNameResolver DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example GCPClusterHostedDNS GatewayAPI ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MetricsCollectionProfiles MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NodeSwap NutanixMultiSubnets OVNObservability OnClusterBuild OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SignatureStores SigstoreImageVerification TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0704 22:24:17.272926 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "ChunkSizeMiB", "CloudDualStackNodeIPs", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "SetEIPForNLBIngressController", "VSphereControlPlaneMachineSet", "VSphereDriverConfiguration", "VSphereMultiVCenters", "VSphereStaticIPs", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AdditionalRoutingCapabilities", "AutomatedEtcdBackup", "BootcNodeManagement", "CSIDriverSharedResource", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ConsolePluginContentSecurityPolicy", "DNSNameResolver", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "GCPClusterHostedDNS", "GatewayAPI", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MetricsCollectionProfiles", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OnClusterBuild", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SignatureStores", "SigstoreImageVerification", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0704 22:24:17.275326 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController I0704 22:24:17.275342 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0704 22:24:17.275370 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0704 22:24:17.275367 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0704 22:24:17.275382 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 22:24:17.275390 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file E0704 22:24:17.275593 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18bf3675dadab9da openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"SetEIPForNLBIngressController\", \"VSphereControlPlaneMachineSet\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"VSphereStaticIPs\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AdditionalRoutingCapabilities\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"CSIDriverSharedResource\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ConsolePluginContentSecurityPolicy\", \"DNSNameResolver\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"GCPClusterHostedDNS\", \"GatewayAPI\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MetricsCollectionProfiles\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OnClusterBuild\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SignatureStores\", \"SigstoreImageVerification\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-07-04 22:24:17.27284681 +0000 UTC m=+0.824192213,LastTimestamp:2026-07-04 22:24:17.27284681 +0000 UTC m=+0.824192213,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" I0704 22:24:17.275604 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0704 22:24:17.275838 1 secure_serving.go:213] Serving securely on [::]:8443 I0704 22:24:17.275865 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" W0704 22:24:17.278526 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0704 22:24:17.278553 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 22:24:17.278627 1 base_controller.go:67] Waiting for caches to sync for ConfigController I0704 22:24:17.283932 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:24:17.283953 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 22:24:17.289684 1 secretconfigobserver.go:119] support secret does not exist I0704 22:24:17.294274 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:24:17.296171 1 recorder.go:156] Pruning old reports every 8h16m38s, max age is 288h0m0s I0704 22:24:17.299674 1 secretconfigobserver.go:119] support secret does not exist I0704 22:24:17.301624 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0704 22:24:17.301636 1 periodic.go:214] Running clusterconfig gatherer I0704 22:24:17.301642 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0704 22:24:17.301653 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0704 22:24:17.301664 1 insightsreport.go:296] Starting report retriever I0704 22:24:17.301669 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0704 22:24:17.301674 1 tasks_processing.go:45] number of workers: 32 I0704 22:24:17.301699 1 tasks_processing.go:69] worker 31 listening for tasks. I0704 22:24:17.301706 1 tasks_processing.go:71] worker 31 working on ingress task. I0704 22:24:17.301750 1 tasks_processing.go:69] worker 15 listening for tasks. I0704 22:24:17.301763 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 22:24:17.301763 1 tasks_processing.go:69] worker 23 listening for tasks. I0704 22:24:17.301772 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 22:24:17.301773 1 tasks_processing.go:69] worker 20 listening for tasks. I0704 22:24:17.301777 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 22:24:17.301775 1 tasks_processing.go:69] worker 19 listening for tasks. I0704 22:24:17.301783 1 tasks_processing.go:69] worker 3 listening for tasks. I0704 22:24:17.301783 1 tasks_processing.go:69] worker 21 listening for tasks. I0704 22:24:17.301789 1 tasks_processing.go:69] worker 18 listening for tasks. I0704 22:24:17.301790 1 tasks_processing.go:69] worker 22 listening for tasks. I0704 22:24:17.301792 1 tasks_processing.go:69] worker 4 listening for tasks. I0704 22:24:17.301793 1 tasks_processing.go:69] worker 17 listening for tasks. I0704 22:24:17.301800 1 tasks_processing.go:69] worker 16 listening for tasks. I0704 22:24:17.301802 1 tasks_processing.go:69] worker 10 listening for tasks. I0704 22:24:17.301800 1 tasks_processing.go:69] worker 9 listening for tasks. I0704 22:24:17.301808 1 tasks_processing.go:69] worker 6 listening for tasks. I0704 22:24:17.301810 1 tasks_processing.go:69] worker 11 listening for tasks. I0704 22:24:17.301808 1 tasks_processing.go:69] worker 7 listening for tasks. I0704 22:24:17.301815 1 tasks_processing.go:69] worker 13 listening for tasks. I0704 22:24:17.301815 1 tasks_processing.go:69] worker 8 listening for tasks. I0704 22:24:17.301820 1 tasks_processing.go:69] worker 5 listening for tasks. I0704 22:24:17.301821 1 tasks_processing.go:69] worker 14 listening for tasks. I0704 22:24:17.301825 1 tasks_processing.go:69] worker 12 listening for tasks. I0704 22:24:17.301826 1 tasks_processing.go:69] worker 25 listening for tasks. I0704 22:24:17.301826 1 tasks_processing.go:69] worker 27 listening for tasks. I0704 22:24:17.301833 1 tasks_processing.go:69] worker 24 listening for tasks. I0704 22:24:17.301833 1 tasks_processing.go:69] worker 30 listening for tasks. I0704 22:24:17.301837 1 tasks_processing.go:69] worker 28 listening for tasks. I0704 22:24:17.301831 1 tasks_processing.go:69] worker 29 listening for tasks. I0704 22:24:17.301845 1 tasks_processing.go:69] worker 26 listening for tasks. I0704 22:24:17.301865 1 tasks_processing.go:71] worker 26 working on sap_config task. I0704 22:24:17.301901 1 tasks_processing.go:71] worker 10 working on infrastructures task. I0704 22:24:17.301909 1 tasks_processing.go:71] worker 1 working on operators task. I0704 22:24:17.301916 1 tasks_processing.go:71] worker 23 working on machine_config_pools task. I0704 22:24:17.301926 1 tasks_processing.go:71] worker 15 working on cluster_apiserver task. I0704 22:24:17.301941 1 tasks_processing.go:71] worker 14 working on nodenetworkstates task. I0704 22:24:17.301950 1 tasks_processing.go:71] worker 20 working on proxies task. I0704 22:24:17.301983 1 tasks_processing.go:71] worker 0 working on machine_autoscalers task. I0704 22:24:17.301989 1 tasks_processing.go:71] worker 3 working on install_plans task. I0704 22:24:17.302032 1 tasks_processing.go:71] worker 9 working on machine_sets task. I0704 22:24:17.302044 1 tasks_processing.go:71] worker 21 working on openstack_dataplanedeployments task. I0704 22:24:17.302046 1 tasks_processing.go:71] worker 2 working on image_registries task. I0704 22:24:17.302116 1 tasks_processing.go:71] worker 17 working on metrics task. I0704 22:24:17.301985 1 tasks_processing.go:71] worker 22 working on active_alerts task. I0704 22:24:17.302175 1 tasks_processing.go:71] worker 25 working on dvo_metrics task. I0704 22:24:17.302211 1 tasks_processing.go:71] worker 18 working on openstack_controlplanes task. I0704 22:24:17.302221 1 tasks_processing.go:71] worker 4 working on version task. I0704 22:24:17.302229 1 tasks_processing.go:71] worker 27 working on aggregated_monitoring_cr_names task. I0704 22:24:17.302254 1 tasks_processing.go:71] worker 16 working on openstack_dataplanenodesets task. I0704 22:24:17.302317 1 tasks_processing.go:71] worker 28 working on feature_gates task. I0704 22:24:17.302402 1 tasks_processing.go:71] worker 6 working on openshift_logging task. I0704 22:24:17.302455 1 tasks_processing.go:71] worker 13 working on cost_management_metrics_configs task. I0704 22:24:17.302170 1 tasks_processing.go:71] worker 12 working on machines task. I0704 22:24:17.302557 1 tasks_processing.go:71] worker 8 working on machine_healthchecks task. I0704 22:24:17.302475 1 tasks_processing.go:71] worker 7 working on pod_network_connectivity_checks task. I0704 22:24:17.302648 1 tasks_processing.go:71] worker 5 working on image task. I0704 22:24:17.301904 1 tasks_processing.go:71] worker 19 working on openshift_machine_api_events task. I0704 22:24:17.302466 1 tasks_processing.go:71] worker 11 working on nodes task. I0704 22:24:17.302482 1 tasks_processing.go:71] worker 24 working on authentication task. I0704 22:24:17.302488 1 tasks_processing.go:71] worker 30 working on node_logs task. I0704 22:24:17.302494 1 tasks_processing.go:71] worker 29 working on support_secret task. E0704 22:24:17.306984 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:34107->172.30.0.10:53: read: connection refused I0704 22:24:17.307024 1 tasks_processing.go:71] worker 17 working on sap_datahubs task. E0704 22:24:17.307038 1 gather.go:144] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:34107->172.30.0.10:53: read: connection refused E0704 22:24:17.307051 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:44158->172.30.0.10:53: read: connection refused I0704 22:24:17.307060 1 gather.go:181] gatherer "clusterconfig" function "metrics" took 4.877143ms to process 0 records E0704 22:24:17.307078 1 gather.go:144] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:44158->172.30.0.10:53: read: connection refused I0704 22:24:17.307090 1 gather.go:181] gatherer "clusterconfig" function "active_alerts" took 4.885419ms to process 0 records I0704 22:24:17.307104 1 tasks_processing.go:71] worker 22 working on operators_pods_and_events task. I0704 22:24:17.310406 1 tasks_processing.go:71] worker 26 working on crds task. I0704 22:24:17.310422 1 gather.go:181] gatherer "clusterconfig" function "sap_config" took 8.527858ms to process 0 records I0704 22:24:17.317368 1 tasks_processing.go:71] worker 21 working on lokistack task. I0704 22:24:17.317408 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 15.314465ms to process 0 records I0704 22:24:17.317466 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkstates" took 15.420835ms to process 0 records I0704 22:24:17.317497 1 gather.go:181] gatherer "clusterconfig" function "openshift_logging" took 14.920746ms to process 0 records I0704 22:24:17.317509 1 gather.go:181] gatherer "clusterconfig" function "machine_autoscalers" took 15.414359ms to process 0 records I0704 22:24:17.317519 1 gather.go:181] gatherer "clusterconfig" function "machine_sets" took 15.36511ms to process 0 records I0704 22:24:17.317525 1 gather.go:181] gatherer "clusterconfig" function "openstack_controlplanes" took 15.186272ms to process 0 records I0704 22:24:17.317549 1 tasks_processing.go:71] worker 14 working on sap_pods task. E0704 22:24:17.317559 1 gather.go:144] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0704 22:24:17.317571 1 gather.go:181] gatherer "clusterconfig" function "machine_healthchecks" took 14.846266ms to process 0 records I0704 22:24:17.317582 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 15.160677ms to process 0 records I0704 22:24:17.317588 1 gather.go:181] gatherer "clusterconfig" function "cost_management_metrics_configs" took 14.993499ms to process 0 records I0704 22:24:17.317596 1 tasks_processing.go:71] worker 13 working on tsdb_status task. I0704 22:24:17.317686 1 tasks_processing.go:71] worker 16 working on olm_operators task. I0704 22:24:17.317761 1 recorder.go:70] Recording config/proxy with fingerprint=41e3b9c3917e7fff5b7c37db08da53f4c466abefcc2ca994a4ae5bda2e148c66 I0704 22:24:17.317780 1 gather.go:181] gatherer "clusterconfig" function "proxies" took 15.673643ms to process 1 records I0704 22:24:17.317794 1 tasks_processing.go:71] worker 0 working on storage_classes task. I0704 22:24:17.317855 1 tasks_processing.go:71] worker 20 working on schedulers task. I0704 22:24:17.317917 1 tasks_processing.go:71] worker 8 working on service_accounts task. I0704 22:24:17.317983 1 tasks_processing.go:71] worker 9 working on silenced_alerts task. I0704 22:24:17.317969 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0704 22:24:17.318140 1 controller.go:317] The initial operator extension status is healthy I0704 22:24:17.318160 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 22:24:17.318168 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0704 22:24:17.318187 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0704 22:24:17.317984 1 tasks_processing.go:71] worker 18 working on machine_configs task. I0704 22:24:17.318214 1 controller.go:457] The operator is still being initialized I0704 22:24:17.318222 1 controller.go:482] The operator is healthy I0704 22:24:17.318632 1 sca.go:98] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/certificates. Next check is in 8h0m0s I0704 22:24:17.318730 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0704 22:24:17.318802 1 operator.go:288] started I0704 22:24:17.318830 1 base_controller.go:67] Waiting for caches to sync for LoggingSyncer I0704 22:24:17.317860 1 tasks_processing.go:71] worker 31 working on container_runtime_configs task. I0704 22:24:17.317990 1 tasks_processing.go:71] worker 6 working on jaegers task. I0704 22:24:17.318143 1 recorder.go:70] Recording config/ingress with fingerprint=dfc7102424a86bb86cbeef35a9dc087303930cc165f8ae4930d7c9abc446facf I0704 22:24:17.319642 1 gather.go:181] gatherer "clusterconfig" function "ingress" took 16.145983ms to process 1 records E0704 22:24:17.319651 1 gather.go:144] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0704 22:24:17.319657 1 gather.go:181] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 15.710152ms to process 0 records I0704 22:24:17.319663 1 gather.go:181] gatherer "clusterconfig" function "openshift_machine_api_events" took 15.615911ms to process 0 records I0704 22:24:17.319739 1 tasks_processing.go:71] worker 5 working on monitoring_persistent_volumes task. I0704 22:24:17.320132 1 tasks_processing.go:71] worker 7 working on image_pruners task. I0704 22:24:17.320456 1 tasks_processing.go:71] worker 19 working on validating_webhook_configurations task. I0704 22:24:17.321022 1 recorder.go:70] Recording config/image with fingerprint=c027bb539472215c1467756c66b0d958f52d942647ef2bab813a589079703018 I0704 22:24:17.321081 1 gather.go:181] gatherer "clusterconfig" function "image" took 15.968457ms to process 1 records I0704 22:24:17.321351 1 tasks_processing.go:71] worker 28 working on oauths task. I0704 22:24:17.321388 1 recorder.go:70] Recording config/featuregate with fingerprint=f8760aa9a61c8eca7ef1086ebcbdeb70cf89b83e7fa81962f3f895e3f5a408df I0704 22:24:17.321585 1 gather.go:181] gatherer "clusterconfig" function "feature_gates" took 16.477189ms to process 1 records I0704 22:24:17.321617 1 gather.go:181] gatherer "clusterconfig" function "sap_datahubs" took 11.826282ms to process 0 records E0704 22:24:17.321646 1 gather.go:144] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0704 22:24:17.321681 1 gather.go:181] gatherer "clusterconfig" function "machines" took 17.614413ms to process 0 records I0704 22:24:17.321706 1 tasks_processing.go:71] worker 12 working on pdbs task. I0704 22:24:17.321689 1 tasks_processing.go:71] worker 17 working on storage_cluster task. I0704 22:24:17.322989 1 tasks_processing.go:71] worker 10 working on config_maps task. I0704 22:24:17.323215 1 recorder.go:70] Recording config/infrastructure with fingerprint=a038f05621563fa889141e557068aa54a8c1cb1cdd05804934b21e2aaa6075a8 I0704 22:24:17.323269 1 gather.go:181] gatherer "clusterconfig" function "infrastructures" took 19.191725ms to process 1 records I0704 22:24:17.323534 1 tasks_processing.go:71] worker 24 working on container_images task. I0704 22:24:17.323771 1 recorder.go:70] Recording config/authentication with fingerprint=0ee0f0ddcae78f030331bf73027a91227cef6cdab83017ca010fc9ab9077d992 I0704 22:24:17.323851 1 gather.go:181] gatherer "clusterconfig" function "authentication" took 18.48025ms to process 1 records E0704 22:24:17.323885 1 gather.go:144] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0704 22:24:17.323940 1 gather.go:181] gatherer "clusterconfig" function "support_secret" took 18.427664ms to process 0 records I0704 22:24:17.323972 1 gather.go:181] gatherer "clusterconfig" function "machine_config_pools" took 20.153289ms to process 0 records I0704 22:24:17.323904 1 tasks_processing.go:71] worker 29 working on mutating_webhook_configurations task. I0704 22:24:17.324330 1 tasks_processing.go:71] worker 23 working on ceph_cluster task. I0704 22:24:17.324779 1 tasks_processing.go:71] worker 2 working on certificate_signing_requests task. I0704 22:24:17.325576 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=0bbec3599e07a3b964d4c84b857bf1702b95355c2dc0e1967935e4e379711ad9 I0704 22:24:17.325598 1 gather.go:181] gatherer "clusterconfig" function "image_registries" took 20.669212ms to process 1 records I0704 22:24:17.325722 1 tasks_processing.go:71] worker 15 working on clusterroles task. I0704 22:24:17.325788 1 recorder.go:70] Recording config/apiserver with fingerprint=ca2c4c69e5542f504633ff77cad10e0f44d401603ffcc0b56b92d5a5ad273e2d I0704 22:24:17.325830 1 gather.go:181] gatherer "clusterconfig" function "cluster_apiserver" took 22.297409ms to process 1 records E0704 22:24:17.326568 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:60343->172.30.0.10:53: read: connection refused I0704 22:24:17.326596 1 tasks_processing.go:71] worker 9 working on nodenetworkconfigurationpolicies task. E0704 22:24:17.326635 1 gather.go:144] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:60343->172.30.0.10:53: read: connection refused I0704 22:24:17.326675 1 gather.go:181] gatherer "clusterconfig" function "silenced_alerts" took 8.556216ms to process 0 records E0704 22:24:17.326678 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:52788->172.30.0.10:53: read: connection refused E0704 22:24:17.326697 1 gather.go:144] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.11:52788->172.30.0.10:53: read: connection refused I0704 22:24:17.326716 1 gather.go:181] gatherer "clusterconfig" function "tsdb_status" took 9.091809ms to process 0 records I0704 22:24:17.326734 1 tasks_processing.go:71] worker 13 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. W0704 22:24:17.328046 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 22:24:17.330532 1 tasks_processing.go:71] worker 21 working on ingress_certificates task. I0704 22:24:17.330578 1 gather.go:181] gatherer "clusterconfig" function "lokistack" took 13.107714ms to process 0 records I0704 22:24:17.330597 1 gather.go:181] gatherer "clusterconfig" function "sap_pods" took 12.968313ms to process 0 records I0704 22:24:17.330603 1 gather.go:181] gatherer "clusterconfig" function "node_logs" took 27.550157ms to process 0 records I0704 22:24:17.330610 1 tasks_processing.go:71] worker 30 working on networks task. I0704 22:24:17.330633 1 tasks_processing.go:71] worker 14 working on overlapping_namespace_uids task. I0704 22:24:17.336523 1 tasks_processing.go:71] worker 0 working on openstack_version task. I0704 22:24:17.336614 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=d1f220fd436ddc3b83179cc7f16dbd375bc65ab741cd30eca897231f32659f51 I0704 22:24:17.336636 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=e909e11b7c98741018fdab26c130dc5ce852d2a6ec7a7caa48983d5b06f3c9b8 I0704 22:24:17.336643 1 gather.go:181] gatherer "clusterconfig" function "storage_classes" took 18.715787ms to process 2 records I0704 22:24:17.340416 1 tasks_processing.go:74] worker 11 stopped. I0704 22:24:17.340732 1 recorder.go:70] Recording config/node/ip-10-0-0-212.ec2.internal with fingerprint=52c137f6b0d7e53237d855ae756a86ab8edcd0eff3f8f907769707a309f8ba36 I0704 22:24:17.340794 1 recorder.go:70] Recording config/node/ip-10-0-1-148.ec2.internal with fingerprint=e40c2968053d8333679ed69b3c959d1e9d0783ac8fbf8c2aebd6f6858e9d2d0c I0704 22:24:17.340852 1 recorder.go:70] Recording config/node/ip-10-0-2-243.ec2.internal with fingerprint=8c2ce91da8167bb4e5ed3187f07142b63ffc25f967808ed5ead0635db278f2c1 I0704 22:24:17.340862 1 gather.go:181] gatherer "clusterconfig" function "nodes" took 37.489561ms to process 3 records I0704 22:24:17.340944 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=ff3b05f87857c599a883af8b5585bbc36ff497d8b5015598ee8269c81a61090b I0704 22:24:17.340959 1 gather.go:181] gatherer "clusterconfig" function "image_pruners" took 20.407802ms to process 1 records I0704 22:24:17.340943 1 tasks_processing.go:74] worker 7 stopped. I0704 22:24:17.341567 1 controller.go:203] Source scaController *sca.Controller is not ready I0704 22:24:17.341583 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0704 22:24:17.341589 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 22:24:17.341594 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0704 22:24:17.341599 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0704 22:24:17.341618 1 controller.go:457] The operator is still being initialized I0704 22:24:17.341623 1 controller.go:482] The operator is healthy I0704 22:24:17.342649 1 tasks_processing.go:74] worker 23 stopped. I0704 22:24:17.342669 1 gather.go:181] gatherer "clusterconfig" function "ceph_cluster" took 18.256555ms to process 0 records I0704 22:24:17.342718 1 gather.go:181] gatherer "clusterconfig" function "storage_cluster" took 19.73851ms to process 0 records I0704 22:24:17.342726 1 gather.go:181] gatherer "clusterconfig" function "container_runtime_configs" took 23.796256ms to process 0 records I0704 22:24:17.342733 1 gather.go:181] gatherer "clusterconfig" function "jaegers" took 23.581471ms to process 0 records I0704 22:24:17.342742 1 tasks_processing.go:74] worker 6 stopped. I0704 22:24:17.342744 1 tasks_processing.go:74] worker 17 stopped. I0704 22:24:17.342748 1 tasks_processing.go:74] worker 31 stopped. I0704 22:24:17.344087 1 tasks_processing.go:74] worker 20 stopped. I0704 22:24:17.344206 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=20c08d7104d50200ed6b3d2d1b327e070ea859c6958e7cfca7f998c383c0d5d5 I0704 22:24:17.344224 1 gather.go:181] gatherer "clusterconfig" function "schedulers" took 26.215779ms to process 1 records I0704 22:24:17.344250 1 tasks_processing.go:74] worker 29 stopped. I0704 22:24:17.344409 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=a7f306efff177ffa004623db28593dbc6faafddc88ea13174786e439b4b8583c I0704 22:24:17.344446 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=245d4bc11ca082844f432a1051e273beb906f513395e6eaf1235d4aaa667c225 I0704 22:24:17.344477 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=d8b61356c7c47daf5993d75d3aa9dd4e13a63272d82d27e99532f61350aa805b I0704 22:24:17.344486 1 gather.go:181] gatherer "clusterconfig" function "mutating_webhook_configurations" took 19.93259ms to process 3 records I0704 22:24:17.344565 1 tasks_processing.go:74] worker 28 stopped. I0704 22:24:17.344694 1 recorder.go:70] Recording config/oauth with fingerprint=8643db7d8824edb98d29ed77f7966968e4abc4c1c3a925ecab292ebbc6202274 I0704 22:24:17.344703 1 gather.go:181] gatherer "clusterconfig" function "oauths" took 22.886894ms to process 1 records I0704 22:24:17.350587 1 tasks_processing.go:74] worker 5 stopped. I0704 22:24:17.350645 1 gather.go:181] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 30.836995ms to process 0 records I0704 22:24:17.352200 1 tasks_processing.go:74] worker 2 stopped. I0704 22:24:17.352259 1 gather.go:181] gatherer "clusterconfig" function "certificate_signing_requests" took 27.39891ms to process 0 records I0704 22:24:17.352321 1 tasks_processing.go:74] worker 12 stopped. I0704 22:24:17.352613 1 recorder.go:70] Recording config/pdbs/openshift-console/console with fingerprint=e743f4e869e7b04e19f5fe3e64f5d45ca7d1f6031c97e9f9d9f4cda5e77f3688 I0704 22:24:17.352696 1 recorder.go:70] Recording config/pdbs/openshift-console/downloads with fingerprint=da33719b7a8baaa8a74c4d14e6ed4a65560cb9f12366929d6cf794324b21a5ad I0704 22:24:17.352760 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=90ab1a1e43d7414bc6f67833994653393a1fb2fb0f468fb7feceebe94f625c4b I0704 22:24:17.352827 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=7b650164c8f20f028a48af6e0a16f3f0fa4e75ecb92bd72e5518fc7627ad55d4 I0704 22:24:17.352889 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=1298b72d0dce27f7f7983d5116ba5a5780daaa3a4b79e1d6bbd9c93ae1a85992 I0704 22:24:17.352953 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=0db98116c204655631bee9aa117463b764b091ee59e56c15b2c03938c0fd8d6f I0704 22:24:17.352988 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=19a87668914ff40cfed71fa91df81508c4a6cc6c0d878ab36bbfc4d004ec777f I0704 22:24:17.353038 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=5f3339a3e40899a0956f3a6b1144ac380d1a87e5e0273aa838ef8393aa9fa1f6 I0704 22:24:17.353145 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=8aa7adf6a4569c1697bb740b4f3b00051737b71b59443b9519c5bee452367997 I0704 22:24:17.353188 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=75ab36b5c2f48eb0818318e73495705d89ce071d4bbf135020bb98d50cdc5b39 I0704 22:24:17.353211 1 gather.go:181] gatherer "clusterconfig" function "pdbs" took 30.527247ms to process 10 records I0704 22:24:17.353241 1 tasks_processing.go:74] worker 0 stopped. I0704 22:24:17.353340 1 gather.go:181] gatherer "clusterconfig" function "openstack_version" took 16.398528ms to process 0 records I0704 22:24:17.353494 1 tasks_processing.go:74] worker 19 stopped. I0704 22:24:17.353887 1 recorder.go:70] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=e32aeb121027d81203cf46c402a8bed42ea574de524db3bc2739f73e4c463c54 I0704 22:24:17.353966 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=64a30ac01eb23da37af353fe53ce3eda2d829c53f2f8d36b4bb5e7dc883d0b8c I0704 22:24:17.354133 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=0a2c581638b29cc158858e1c584ded0ec91b8325ac08b1b74d2c9e7d5885e8db I0704 22:24:17.354174 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=b302276b3aa063c857e370b24bdde744c23c921e9a7107024d62922f5b8a28d1 I0704 22:24:17.354205 1 recorder.go:70] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=de03306ab438852e028dacbcb6abb532004ec3aefa43eeceab5451c33549f9e6 I0704 22:24:17.354240 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=bcc86124221e8b4ca504393e2e6a26bda139e9cef99523fda3725abf09749f27 I0704 22:24:17.354284 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=cc01098572a6c5c86e9e8f648a14d3103922cb1da00d3c90c2193bd3a2836609 I0704 22:24:17.354338 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=8fa08d2b9c2ec58d3fcb5190669a21d8893e07c2deeeab35462793847dcbf223 I0704 22:24:17.354379 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=65b60066f20010dc7d4c3fb0135642574418b781030174ad2d59817c0032009d I0704 22:24:17.354420 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=7f57dd8e2dde149dd9fc9463b4e779cfe45328ce304836cff87289b69c5a2103 I0704 22:24:17.354449 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=f1aaa9a5d870cb0346c12836297378e26b1b8105e0bc16034fedaa86d5a865ac I0704 22:24:17.354480 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=d21ae4f5d14703b1c113e19f0a2614432a93162d7d16e9f80b45e6bf936ed4ac I0704 22:24:17.354513 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=859881a4491f7fe68e7c8908ce7a47d6d9498347c36bb588fef535d50240bcea I0704 22:24:17.354522 1 gather.go:181] gatherer "clusterconfig" function "validating_webhook_configurations" took 32.988723ms to process 13 records I0704 22:24:17.354544 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 27.506949ms to process 0 records I0704 22:24:17.354561 1 tasks_processing.go:74] worker 9 stopped. E0704 22:24:17.355575 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%!c(MISSING)fd0da7-ea95-4765-aae4-2f77db80a3a5%!+(MISSING)and+status+is+%!a(MISSING)ccepted%!"(MISSING): dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:47164->172.30.0.10:53: read: connection refused I0704 22:24:17.355593 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%272cfd0da7-ea95-4765-aae4-2f77db80a3a5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:47164->172.30.0.10:53: read: connection refused I0704 22:24:17.358472 1 requests.go:204] Asking for SCA certificate for x86_64 architecture W0704 22:24:17.361421 1 sca.go:117] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:48396->172.30.0.10:53: read: connection refused I0704 22:24:17.361433 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:48396->172.30.0.10:53: read: connection refused I0704 22:24:17.361963 1 tasks_processing.go:74] worker 18 stopped. E0704 22:24:17.361980 1 gather.go:144] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0704 22:24:17.362042 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0704 22:24:17.362067 1 gather.go:181] gatherer "clusterconfig" function "machine_configs" took 43.752145ms to process 1 records I0704 22:24:17.362389 1 tasks_processing.go:74] worker 30 stopped. I0704 22:24:17.362632 1 recorder.go:70] Recording config/network with fingerprint=57aa4dcd338c546e703f153df7de29bada11850b074b3a073b79e39b3530fb44 I0704 22:24:17.362646 1 gather.go:181] gatherer "clusterconfig" function "networks" took 31.758935ms to process 1 records I0704 22:24:17.363063 1 tasks_processing.go:74] worker 4 stopped. I0704 22:24:17.363512 1 recorder.go:70] Recording config/version with fingerprint=28e71894ead4b060897a5e2b40d68444fbade1dbca484a56b02330936c5d6cab I0704 22:24:17.363525 1 recorder.go:70] Recording config/id with fingerprint=9e39f45171723ee7959afd7012943afcc93e337e9be1e0fe748f5c066f7a7af5 I0704 22:24:17.363531 1 gather.go:181] gatherer "clusterconfig" function "version" took 60.831014ms to process 2 records I0704 22:24:17.364920 1 tasks_processing.go:74] worker 14 stopped. I0704 22:24:17.364943 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0704 22:24:17.364952 1 gather.go:181] gatherer "clusterconfig" function "overlapping_namespace_uids" took 34.269281ms to process 1 records I0704 22:24:17.365116 1 tasks_processing.go:74] worker 27 stopped. I0704 22:24:17.365133 1 gather.go:181] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 62.871346ms to process 0 records I0704 22:24:17.366529 1 tasks_processing.go:74] worker 16 stopped. I0704 22:24:17.366599 1 recorder.go:70] Recording config/olm_operators with fingerprint=91a2ed24633b784e7118c7079768650f3758842f04580dab8549d258c6db7b47 I0704 22:24:17.366609 1 gather.go:181] gatherer "clusterconfig" function "olm_operators" took 48.824516ms to process 1 records I0704 22:24:17.366934 1 tasks_processing.go:74] worker 26 stopped. I0704 22:24:17.367697 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=4b9e213ad3086ad5dd3df4d689a39098eccf7ba61e890a62a3ecd2cde0e3cfcf I0704 22:24:17.367917 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=1f4280513b3aef49c118b088ad30a23d189c553245520345fb305db52891d7bb I0704 22:24:17.367931 1 gather.go:181] gatherer "clusterconfig" function "crds" took 56.512545ms to process 2 records I0704 22:24:17.368452 1 tasks_processing.go:74] worker 15 stopped. I0704 22:24:17.369166 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=7ab423e0b62e94c96999f149f14e1c1f51d368b33bf3aab53682cef3b1ad6a79 I0704 22:24:17.369355 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=41bbdbd32fbb61763a3bba21cfd5645cf5e625804ddadb4b7ca2f37fb5e1c6be I0704 22:24:17.369391 1 gather.go:181] gatherer "clusterconfig" function "clusterroles" took 42.704838ms to process 2 records I0704 22:24:17.375439 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0704 22:24:17.375448 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 22:24:17.375439 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 22:24:17.378714 1 base_controller.go:73] Caches are synced for ConfigController I0704 22:24:17.378771 1 base_controller.go:110] Starting #1 worker of ConfigController controller ... I0704 22:24:17.379549 1 tasks_processing.go:74] worker 24 stopped. I0704 22:24:17.380993 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz with fingerprint=2f5f69dc62b30eadc3b559ab1f66c423c0fb991c587115c0789ec0421581aadc I0704 22:24:17.381263 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-9js54 with fingerprint=2ec4e536ee6475b3d7ac32010f2f577601016a3a37610a84366c3d0a71563f42 I0704 22:24:17.381476 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-s8m8q with fingerprint=57c90d6ba9794aba787bc5099d2a75be1f0be6c6d1df1f7fcbac4e21e3a29efd I0704 22:24:17.381576 1 recorder.go:70] Recording config/running_containers with fingerprint=d3b08a80e013abc7e9c6fa84ab2e55ba5523337e7f3cb82ca5953252b218286a I0704 22:24:17.381587 1 gather.go:181] gatherer "clusterconfig" function "container_images" took 55.991115ms to process 4 records I0704 22:24:17.391415 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 22:24:17.392326 1 tasks_processing.go:74] worker 13 stopped. I0704 22:24:17.392339 1 gather.go:181] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 65.574591ms to process 0 records I0704 22:24:17.401069 1 tasks_processing.go:74] worker 10 stopped. E0704 22:24:17.401086 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0704 22:24:17.401094 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0704 22:24:17.401097 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0704 22:24:17.401136 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0704 22:24:17.401146 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=e1a0ad594db3a5d5d50e9e25d7b235af20f84f5def41537f127febe525b834f2 I0704 22:24:17.401151 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0704 22:24:17.401160 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0704 22:24:17.401197 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0704 22:24:17.401205 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0704 22:24:17.401210 1 gather.go:181] gatherer "clusterconfig" function "config_maps" took 78.036811ms to process 6 records I0704 22:24:17.418997 1 base_controller.go:73] Caches are synced for LoggingSyncer I0704 22:24:17.419039 1 base_controller.go:110] Starting #1 worker of LoggingSyncer controller ... I0704 22:24:17.431411 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress-operator" I0704 22:24:17.482752 1 tasks_processing.go:74] worker 21 stopped. E0704 22:24:17.482769 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0704 22:24:17.482776 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rbj8kgb7eavt1krbjkdvkkldp2dihm3-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rbj8kgb7eavt1krbjkdvkkldp2dihm3-primary-cert-bundle-secret" not found I0704 22:24:17.482842 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=b52d64109927f4aa3de91e5a97123e87a4c1908eaf4e8a636ed24db31fe2fd34 I0704 22:24:17.482855 1 gather.go:181] gatherer "clusterconfig" function "ingress_certificates" took 152.200316ms to process 1 records I0704 22:24:17.977662 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0704 22:24:18.177078 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" W0704 22:24:18.325564 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 22:24:18.537622 1 gather_cluster_operator_pods_and_events.go:121] Found 24 pods with 27 containers I0704 22:24:18.537639 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 932067 bytes I0704 22:24:18.537819 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-7d4ddc79d6-vzqqz pod in namespace openshift-console-operator (previous: true). I0704 22:24:18.744275 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-7d4ddc79d6-vzqqz pod in namespace openshift-console-operator (previous: false). I0704 22:24:18.947509 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-66bb5cccd4-rh8vb pod in namespace openshift-console (previous: false). I0704 22:24:18.980135 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0704 22:24:19.143929 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-66bb5cccd4-sjxvd pod in namespace openshift-console (previous: false). W0704 22:24:19.326033 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 22:24:19.343811 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-699bb6bbbc-j4mxz pod in namespace openshift-console (previous: false). I0704 22:24:19.545994 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-6d4c9b4b6c-88gs9 pod in namespace openshift-console (previous: false). I0704 22:24:19.583114 1 tasks_processing.go:74] worker 1 stopped. I0704 22:24:19.583282 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=24ab349d12df0bac068f530047c14383496c30897dc22aa133ef3e53fd295222 I0704 22:24:19.583340 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=183539bbe89c989a326011f124cb26e91d4dc0406689baecde44742b1c4a8e80 I0704 22:24:19.583366 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=53007952fa3518facf078c7138d3bb1221212a431c93bb504d7791550d22c573 I0704 22:24:19.583378 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0704 22:24:19.583406 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=8f9d7f95996d5c869e59749b08f789cc217251f98de0bf706c6a5522628067a0 I0704 22:24:19.583423 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0704 22:24:19.583446 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=1038e9d68c09f08335c0d6bcd76f89b9a4379adcae15d1f0f2e40c6cc8cb6c6d I0704 22:24:19.583479 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=cc692d1e210a0866750e4f8440b474e463ff607d511e78767bfb013cdcf3df99 I0704 22:24:19.583505 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=036aaabafd116da0cf8047118b9a121b0a4792491436fb6731cde171dac16005 I0704 22:24:19.583538 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=edc58b417f101a9860a5749d3d0156441edc96014b17f4fb9a2e172dca1bafcb I0704 22:24:19.583889 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=b9e3ad2ad34e310aeccab341cd0760125707f6aef11e4c2504cfb1871573a09d I0704 22:24:19.583915 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=0c95dacab4e7648b1ffe745255c3220a8cdd32ab6c899c85516548f80855c204 I0704 22:24:19.583929 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0704 22:24:19.583950 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=31a1c5f94823ab332a23a70e07259085125c6f9a683ddde64379fe876be9c7b8 I0704 22:24:19.583962 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0704 22:24:19.583978 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=023dc3a8aacd972fbdbec0e9d6839a21733b14a7c08e1b6beaa3db5b82c4a842 I0704 22:24:19.583988 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0704 22:24:19.584027 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=35fab882285036ef689629859d95192427786626bf9ad6f4a310a720f453dd45 I0704 22:24:19.584038 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0704 22:24:19.584059 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=ac0bfc1e5006fd0cacb824415a3f25321aa298a0e1de046d9c6ea4e6ef16e76e I0704 22:24:19.584170 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=b43bde8e98fb7a932f89a4744436c1a365a29f8311f9496fc0a8b0ec854e670b I0704 22:24:19.584205 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0704 22:24:19.584228 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=50036b6c9d89736c30eb9b903afa490fe144636cd061ed5f12ccf1762637da2e I0704 22:24:19.584255 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=43858cc3474d70b1ffc5e639dd6e6a00eda43b4b4944ce86f12b4ba42f0a0f02 I0704 22:24:19.584267 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0704 22:24:19.584285 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=cac14f1944cbe782cc358f5d9be105729b8ee430fdb51fffb703c3bb8c933674 I0704 22:24:19.584294 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0704 22:24:19.584308 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=23e07ef26580ea5650ada81aea1abece73e869ff6f11af8257b29c6651cb977f I0704 22:24:19.584322 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=89f09795e6d423d43b0216ae93237aa71c55fa30d4389d484aba59a8fd842ad1 I0704 22:24:19.584345 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=f89ad7a3bc7534d7216dc3a03adf628fe0f57606c0adfd2e513b3418aa5aaf3e I0704 22:24:19.584362 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=dee433af450c2f21c2e4484c9245c4f9deeb78d9230be128620179de0138fd49 I0704 22:24:19.584383 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=bb59be61d10f9ab33acad0995661a9a8ab56bcce9a0b028759eac7d4d6a2a2a9 I0704 22:24:19.584394 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0704 22:24:19.584427 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=54f5fa43ef020f80469fab4352ea74aefd0991da1d43344e97c6cdaca3215d4c I0704 22:24:19.584446 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0704 22:24:19.584455 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0704 22:24:19.584462 1 gather.go:181] gatherer "clusterconfig" function "operators" took 2.281190287s to process 36 records I0704 22:24:19.743585 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-6d4c9b4b6c-jsg4p pod in namespace openshift-console (previous: false). I0704 22:24:19.952393 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-mcwxj pod in namespace openshift-dns (previous: false). I0704 22:24:20.142807 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mcwxj pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-mcwxj\" is waiting to start: ContainerCreating" I0704 22:24:20.142829 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-mcwxj\" is waiting to start: ContainerCreating" I0704 22:24:20.142838 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-mcwxj pod in namespace openshift-dns (previous: false). W0704 22:24:20.326121 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 22:24:20.342799 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mcwxj pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-mcwxj\" is waiting to start: ContainerCreating" I0704 22:24:20.342814 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-mcwxj\" is waiting to start: ContainerCreating" I0704 22:24:20.342831 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-tkqmg pod in namespace openshift-dns (previous: false). I0704 22:24:20.544270 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-tkqmg pod in namespace openshift-dns (previous: false). I0704 22:24:20.744685 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-wm8fk pod in namespace openshift-dns (previous: false). I0704 22:24:20.944650 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-wm8fk pod in namespace openshift-dns (previous: false). I0704 22:24:21.143835 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8zn5l pod in namespace openshift-dns (previous: false). W0704 22:24:21.326134 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0704 22:24:21.345245 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 22:24:21.345263 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-hgskl pod in namespace openshift-dns (previous: false). I0704 22:24:21.544089 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 22:24:21.544110 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-qqxqn pod in namespace openshift-dns (previous: false). I0704 22:24:21.743990 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 22:24:21.744022 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-554b97db8b-nqfvs pod in namespace openshift-image-registry (previous: false). I0704 22:24:21.946066 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5b6df889c8-rtl9k pod in namespace openshift-image-registry (previous: false). I0704 22:24:22.144917 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5b6df889c8-x2h4v pod in namespace openshift-image-registry (previous: false). W0704 22:24:22.326519 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0704 22:24:22.326547 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0704 22:24:22.326565 1 tasks_processing.go:74] worker 25 stopped. E0704 22:24:22.326584 1 gather.go:144] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0704 22:24:22.326603 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0704 22:24:22.326619 1 gather.go:181] gatherer "clusterconfig" function "dvo_metrics" took 5.024375442s to process 1 records I0704 22:24:22.344516 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9jn76 pod in namespace openshift-image-registry (previous: false). I0704 22:24:22.543652 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 22:24:22.543669 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-qfc72 pod in namespace openshift-image-registry (previous: false). I0704 22:24:22.743728 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-s72kr pod in namespace openshift-image-registry (previous: false). I0704 22:24:22.944854 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-68b5fbd886-4v8zj pod in namespace openshift-ingress (previous: false). I0704 22:24:23.147677 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-68b5fbd886-88fgs pod in namespace openshift-ingress (previous: false). I0704 22:24:23.344901 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-b879f4c47-pdfcq pod in namespace openshift-ingress (previous: false). I0704 22:24:23.545729 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-9tzkh pod in namespace openshift-ingress-canary (previous: false). I0704 22:24:23.744883 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-hntpb pod in namespace openshift-ingress-canary (previous: false). I0704 22:24:23.946766 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-t4vzp pod in namespace openshift-ingress-canary (previous: false). I0704 22:24:24.143275 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-t4vzp pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t4vzp\" is waiting to start: ContainerCreating" I0704 22:24:24.143292 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t4vzp\" is waiting to start: ContainerCreating" I0704 22:24:24.143316 1 tasks_processing.go:74] worker 22 stopped. I0704 22:24:24.143508 1 recorder.go:70] Recording events/openshift-console-operator with fingerprint=f500d790e49a4b50ea5fcac20f5fd8c7b891d3f2e0d75252c066d3cbfcbb7424 I0704 22:24:24.143620 1 recorder.go:70] Recording events/openshift-console with fingerprint=0b5700fcf532af52d8ca799bbf16be217c60b236b3a14c8905fb5ad173d1b2bb I0704 22:24:24.143642 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=82d5c0985ea7ed927625748eaca86ae0e003f5eef10431cb83bfa4c02d495722 I0704 22:24:24.143708 1 recorder.go:70] Recording events/openshift-dns with fingerprint=691838b033c8d7a1835fc6a4eea77bbe4b5c65e705048fde5a0078f74f1b92cd I0704 22:24:24.143851 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=3011aef5ee73a9910ce515f734f8a4988595ae21cf426af9906edae491de38d3 I0704 22:24:24.143884 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=cfdb426802ebdd709302aeb9983698a7371ffdfde3549c55baa7e84c186a98ee I0704 22:24:24.143956 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=23263af378dfd07f09fc0916f9cfc9178c8a004fa01d89a43812a78587865af1 I0704 22:24:24.143990 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=b94f0dc225a317777224170e338531eb7def8acb36608cc11f0baf3dd964c148 I0704 22:24:24.144137 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz with fingerprint=2f5f69dc62b30eadc3b559ab1f66c423c0fb991c587115c0789ec0421581aadc E0704 22:24:24.144157 1 gather.go:165] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json" was already recorded and had the fingerprint "2f5f69dc62b30eadc3b559ab1f66c423c0fb991c587115c0789ec0421581aadc", overwriting with the record having fingerprint "2f5f69dc62b30eadc3b559ab1f66c423c0fb991c587115c0789ec0421581aadc" W0704 22:24:24.144168 1 gather.go:159] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json" because of the warning: warning: the record with the same fingerprint "2f5f69dc62b30eadc3b559ab1f66c423c0fb991c587115c0789ec0421581aadc" was already recorded at path "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json" I0704 22:24:24.144190 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-7d4ddc79d6-vzqqz/console-operator_previous.log with fingerprint=474f9712d4d2656da752c6620cad9e662be0b1b3b6c80bfa6f48ad631253a546 I0704 22:24:24.144316 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-7d4ddc79d6-vzqqz/console-operator_current.log with fingerprint=1c9fc6bf3917fe91985d2058badf0f618b06dd2cd71a3ec6a4439bfb95eb4127 I0704 22:24:24.144326 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-66bb5cccd4-rh8vb/console_current.log with fingerprint=75a8ce7eb4b9e86edb19a561251d64df173f6a0649d2b8452bbf61be33a4e28a I0704 22:24:24.144334 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-66bb5cccd4-sjxvd/console_current.log with fingerprint=5312e9c1a014482b42bd6e20158ca32c1f1439cbb9f53a93923ed6fa153fb182 I0704 22:24:24.144341 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-699bb6bbbc-j4mxz/console_current.log with fingerprint=fcf17d46dc6ef755fd3f313b98c4fc2f49c248dfdd74f314027ef46891b9bb8e I0704 22:24:24.144348 1 recorder.go:70] Recording config/pod/openshift-console/logs/downloads-6d4c9b4b6c-88gs9/download-server_current.log with fingerprint=1907ec3bd6c7bbbc943ec048df74aad4504a5c72e5df189cbdc3067e35725616 I0704 22:24:24.144369 1 recorder.go:70] Recording config/pod/openshift-console/logs/downloads-6d4c9b4b6c-jsg4p/download-server_current.log with fingerprint=36a66bafe9d30d40d2e591d1aaa93cc5bb27a7c7f57c75dfafc57da50f525459 I0704 22:24:24.144458 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-mcwxj with fingerprint=df4d4f0c974ba2a5e85930d33c3ed905ad0a72365339d5f9a694726294965f43 I0704 22:24:24.144467 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-tkqmg/dns_current.log with fingerprint=6ea0cf6c955fedc8fa313f4b896db285a95d09dc356272b5169db79e8d7dc68e I0704 22:24:24.144475 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-tkqmg/kube-rbac-proxy_current.log with fingerprint=ff2a195ce7813a87e547eae98dab32b33d5ca0429a2f074ab61804ce34ee2fcd I0704 22:24:24.144481 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-wm8fk/dns_current.log with fingerprint=1603694f387abfab66c545f410ac46a570a569cf984627d814748ae14daa0f6a I0704 22:24:24.144488 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-wm8fk/kube-rbac-proxy_current.log with fingerprint=6d068f800a7d47a9c2541cfe0e222e5fa998ee6bde9b41361de585ab0f034829 I0704 22:24:24.144525 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-554b97db8b-nqfvs/registry_current.log with fingerprint=a41b2b44d1cd39cdab1f2cabc44035ba17d25869d944b289a36c26206785e2c2 I0704 22:24:24.144552 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-5b6df889c8-rtl9k/registry_current.log with fingerprint=493cd40aa395410922a94150c6fd9449bb6a52739477e15c5c55781928c88b3f I0704 22:24:24.144590 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-5b6df889c8-x2h4v/registry_current.log with fingerprint=f29ddad54e4c6a1e45066676395332872d5953fd28922a52e12c50c8071b2d3f I0704 22:24:24.144598 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/node-ca-qfc72/node-ca_current.log with fingerprint=5b4ff0a1c27ea28fc270c5c11e763d65add6570c19a4485b191bb105e9ed4838 I0704 22:24:24.144604 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/node-ca-s72kr/node-ca_current.log with fingerprint=90e28d03357635430338d79060a257b60cedf66e2d00358d2cce51db35dd1fd5 I0704 22:24:24.144619 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-68b5fbd886-4v8zj/router_current.log with fingerprint=4ce78cf081b79a40fd87c1c6a034e87d41c08e1563ac2992408fc71a9e706a6c I0704 22:24:24.144636 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-68b5fbd886-88fgs/router_current.log with fingerprint=12110dd527ce0e62a9f966991ce5f0026c41f8370643193e58bcafe2427006f6 I0704 22:24:24.144654 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-b879f4c47-pdfcq/router_current.log with fingerprint=59d5e9e74515b062a8cde284b16010e04e52bf9dfa981197dd28eb091754a815 I0704 22:24:24.144661 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-9tzkh/serve-healthcheck-canary_current.log with fingerprint=a3a2b073818d7875e4a5130ce4805aac78d533dd20740865daccf8fc9d36be5d I0704 22:24:24.144666 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-hntpb/serve-healthcheck-canary_current.log with fingerprint=41edfce76f8a6dbe3483c37a95ce5fc4947468f396340ba8280c85772dc8031c I0704 22:24:24.144722 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-t4vzp with fingerprint=843ba4160217ca04de4acd70f5fe170d3039cf98bb95f67d367098627bc96966 I0704 22:24:24.144731 1 gather.go:181] gatherer "clusterconfig" function "operators_pods_and_events" took 6.836198222s to process 32 records I0704 22:24:30.330796 1 tasks_processing.go:74] worker 3 stopped. I0704 22:24:30.330838 1 recorder.go:70] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0704 22:24:30.330852 1 gather.go:181] gatherer "clusterconfig" function "install_plans" took 13.028787953s to process 1 records I0704 22:24:31.124944 1 tasks_processing.go:74] worker 8 stopped. I0704 22:24:31.125240 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=ba16ea6b4503917d33437d4d1999039313b5aa3c669ab94c714d5681918865aa I0704 22:24:31.125262 1 gather.go:181] gatherer "clusterconfig" function "service_accounts" took 13.806948293s to process 1 records E0704 22:24:31.125312 1 periodic.go:252] clusterconfig failed after 13.823s with: function "metrics" failed with an error, function "active_alerts" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json" I0704 22:24:31.125325 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "metrics" failed with an error, function "active_alerts" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-vzqqz.json" I0704 22:24:31.125331 1 periodic.go:214] Running workloads gatherer I0704 22:24:31.125343 1 tasks_processing.go:45] number of workers: 2 I0704 22:24:31.125352 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 22:24:31.125356 1 tasks_processing.go:71] worker 1 working on workload_info task. I0704 22:24:31.125368 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 22:24:31.125417 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0704 22:24:31.150543 1 gather_workloads_info.go:281] Loaded pods in 0s, will wait 24s for image data I0704 22:24:31.158342 1 tasks_processing.go:74] worker 0 stopped. I0704 22:24:31.158357 1 gather.go:181] gatherer "workloads" function "helmchart_info" took 32.905907ms to process 0 records I0704 22:24:31.158674 1 gather_workloads_info.go:390] No image sha256:0c1abef76d6d2832ddcff31e950ffafeb4e0f6fad80a5f5dca94591de0fc9581 (9ms) I0704 22:24:31.167399 1 gather_workloads_info.go:390] No image sha256:5bcfb1b739fa4e6ff9e21e4843f8520f29608135341f7d8dcc31084b9ba25f1d (9ms) I0704 22:24:31.175697 1 gather_workloads_info.go:390] No image sha256:1eee856bde9bad71c4444c494225a0cf02d1f4a5293acf0bd6e7e1814efa0064 (8ms) I0704 22:24:31.184079 1 gather_workloads_info.go:390] No image sha256:4b9b7949d209e257a76053de4ac74193f747c0d42e25dcd270d655849a4e51aa (8ms) I0704 22:24:31.192468 1 gather_workloads_info.go:390] No image sha256:3fea7db586d3236c1876a8b523af0d8d65c84d6f6cf5d0f4e41cafcffe6db5b5 (8ms) I0704 22:24:31.200323 1 gather_workloads_info.go:390] No image sha256:1c7c24111f1709d7648156873ced09462638aa4b3cf7e7aee13297e51bcdfb3e (8ms) I0704 22:24:31.208518 1 gather_workloads_info.go:390] No image sha256:c7aa910ca06f8fd89cdf2519136e0f3533bdacfc6bd58a3f6683f9f08756c7b1 (8ms) I0704 22:24:31.218273 1 gather_workloads_info.go:390] No image sha256:02283a64c1936a5dd09d5c6bc0ecbcf9196fe4df6a4055c8be22cebd850aa7a3 (10ms) I0704 22:24:31.228434 1 gather_workloads_info.go:390] No image sha256:30ff5e5b72f274598704ca69f1832eba1040cc25da47386004fd59500b6e4c55 (10ms) I0704 22:24:31.236356 1 gather_workloads_info.go:390] No image sha256:09b3e9e6a4f382fb4bbdb67bc945bcf19d66f604b7c8ffd66d4d4a2c94224248 (8ms) I0704 22:24:31.257414 1 gather_workloads_info.go:390] No image sha256:3a2c82b772ea6072e1eca0efc5303f002189e80b0e985fb3e3b8ea9a12fe8060 (21ms) I0704 22:24:31.359595 1 gather_workloads_info.go:390] No image sha256:894452e6587de55d04e37163d3da53e11fced8fb08bd0acf61e6d49f7964284e (102ms) I0704 22:24:31.460926 1 gather_workloads_info.go:390] No image sha256:751bd7eb62f632c0eef0ca91b6c57c64d4f1279d4dbe1a9b875ac4020a7b693f (101ms) I0704 22:24:31.559536 1 gather_workloads_info.go:390] No image sha256:13aabbaf260d634ff0e9ac1af88af075b5ba671fd51313e33e74d7d2f7e54128 (99ms) I0704 22:24:31.659500 1 gather_workloads_info.go:390] No image sha256:24519dcc0e63cdc60a639dd474d06e7612e9b76966bc60322cd9026a0f1b878c (100ms) I0704 22:24:31.759223 1 gather_workloads_info.go:390] No image sha256:42b4c6bdf3081ec264bc5edbc4eb779151de9e4742d6fe59a316645891df649c (100ms) I0704 22:24:31.862460 1 gather_workloads_info.go:390] No image sha256:38787de16ee8c686305f93b501d891b50cde24136009863869d230c0d4eeee5c (103ms) I0704 22:24:31.959502 1 gather_workloads_info.go:390] No image sha256:31a22f6ee061d0f6558664e580ec0f80f1927d4ec29626cf3e9420837f5d80e5 (97ms) I0704 22:24:32.059354 1 gather_workloads_info.go:390] No image sha256:4dcdadc14b7bdb2c00954d92e957d63aafbbe5f4047fe028e575d65725923cb0 (100ms) I0704 22:24:32.159432 1 gather_workloads_info.go:390] No image sha256:221622a101db9797df0622b4534e3e42c4498df458f355a8a02fad8d65f8e3f2 (100ms) I0704 22:24:32.260729 1 gather_workloads_info.go:390] No image sha256:586127c49e25052113263d8aeaf6a1623aef22987364f119a17b2848982ff716 (101ms) I0704 22:24:32.363345 1 gather_workloads_info.go:390] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (103ms) I0704 22:24:32.459476 1 gather_workloads_info.go:390] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (96ms) I0704 22:24:32.560415 1 gather_workloads_info.go:390] No image sha256:8562a483f4568254250f19fc68d79f88c94a48508aefe59a70c1021a3aad000a (101ms) I0704 22:24:32.661723 1 gather_workloads_info.go:390] No image sha256:194ee1155b87e110bc6c4a6e55667e41f3bdc8292e26a9561f9082fdfe467eec (101ms) I0704 22:24:32.758773 1 gather_workloads_info.go:390] No image sha256:b8781637560d49de9b8e8cdd59ec75f03d98921ea66950381849398128cc9cb9 (97ms) I0704 22:24:32.859246 1 gather_workloads_info.go:390] No image sha256:0158b297aec0a66376c04d3c7cf4828db5e5d9f424e331ff7c7b796828036875 (100ms) I0704 22:24:32.958474 1 gather_workloads_info.go:390] No image sha256:a7fc4d05c4a3c82c9f355ca985d6fd26033c0d6a98716f25fcd2f54bae1d2fa0 (99ms) I0704 22:24:33.061106 1 gather_workloads_info.go:390] No image sha256:50076d29400ecf9b704d8175509fc8cb7c28bea91d46a28dc37cbda1668c69f9 (103ms) I0704 22:24:33.159268 1 gather_workloads_info.go:390] No image sha256:2897724cec17bc13acc133181f64ccb8d54ce5dc9582f9cecf704c2540e15600 (98ms) I0704 22:24:33.258442 1 gather_workloads_info.go:390] No image sha256:3969660e807e8ce5656a74e5816573656241946ef1d766a9d2e21d434356313c (99ms) I0704 22:24:33.358374 1 gather_workloads_info.go:390] No image sha256:41029e57ece558b2e0b12174ce924647fd9d5c5d83aadf4bba03d6916d3f9f54 (100ms) I0704 22:24:33.460841 1 gather_workloads_info.go:390] No image sha256:48dff3fcb0da58eb9b2b500f89af8b7d0790ffa60da0b0e301feca971dbafcbb (102ms) I0704 22:24:33.559466 1 gather_workloads_info.go:390] No image sha256:6a9b4fd887fa7f1729302fb0330c3c9a483197bf78becdcfda4235ec823a2c39 (99ms) I0704 22:24:33.661023 1 gather_workloads_info.go:390] No image sha256:bc829520eeec090b60d17abbad1a22b34ea99b426193395808a778d73a5fb57a (102ms) I0704 22:24:33.759536 1 gather_workloads_info.go:390] No image sha256:68abf16a3b6c33fdd22a823c7d00f96aaaa8015d370116962a5835984908645a (98ms) I0704 22:24:33.861548 1 gather_workloads_info.go:390] No image sha256:788d16bfff4bd6bb3ce97acc643b56fae6e619674ab279535480e519b4d5b769 (102ms) I0704 22:24:33.963974 1 gather_workloads_info.go:390] No image sha256:0c6b7f828184b16a5e6630111e0715e11d34dae9b0bf64619466543e6168e05c (102ms) I0704 22:24:34.059581 1 gather_workloads_info.go:390] No image sha256:c96b4bddf3dd7daaa797cbda4dd92e981de2f5fc3404c01381f31dc0414d5c8d (96ms) I0704 22:24:34.159148 1 gather_workloads_info.go:390] No image sha256:ed0c9631d6e424912cd8ce78ca7a2c4f76b3fc15e9e34fe00999170ccf19a611 (100ms) I0704 22:24:34.260627 1 gather_workloads_info.go:390] No image sha256:06821dfe231330e0eb5be915b9468cd13826a7ee59d1a732b89cb140548eade2 (101ms) I0704 22:24:34.359041 1 gather_workloads_info.go:390] No image sha256:351ca92471647f383f3cd7f0b70f33f5fa11c39ab569c4948dff7d471a6c8be3 (98ms) I0704 22:24:34.458969 1 gather_workloads_info.go:390] No image sha256:0242cefbf0bb876101c4d67ab4e07401cb0ba6c02be2b73a4f17ba33b2960005 (100ms) I0704 22:24:34.559172 1 gather_workloads_info.go:390] No image sha256:8e158c83ec12b3e0237fab9a221630bd691e79c079a111f346354ff43b8199df (100ms) I0704 22:24:34.559206 1 tasks_processing.go:74] worker 1 stopped. I0704 22:24:34.559517 1 recorder.go:70] Recording config/workload_info with fingerprint=9d7d0eff62cd250a439730234a58448e163ea5e513cefc606e51081ae1e6ed5d I0704 22:24:34.559533 1 gather.go:181] gatherer "workloads" function "workload_info" took 3.433842082s to process 1 records I0704 22:24:34.559547 1 periodic.go:261] Periodic gather workloads completed in 3.434s I0704 22:24:34.559556 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0704 22:24:34.559561 1 periodic.go:214] Running conditional gatherer I0704 22:24:34.568796 1 requests.go:282] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.18.9/gathering_rules I0704 22:24:34.825054 1 conditional_gatherer.go:336] updating alerts cache for conditional gatherer E0704 22:24:34.827443 1 conditional_gatherer.go:322] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.117.228:9091: connect: connection refused I0704 22:24:34.827517 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0704 22:24:34.836855 1 conditional_gatherer.go:390] cluster version is '4.18.9' E0704 22:24:34.836871 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836877 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836881 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836885 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836888 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836892 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836895 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836898 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836901 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0704 22:24:34.836905 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0704 22:24:34.836919 1 tasks_processing.go:45] number of workers: 3 I0704 22:24:34.836927 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 22:24:34.836932 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0704 22:24:34.836937 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 22:24:34.836949 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 22:24:34.836955 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0704 22:24:34.836958 1 tasks_processing.go:74] worker 1 stopped. I0704 22:24:34.836951 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0704 22:24:34.837039 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0704 22:24:34.837050 1 gather.go:181] gatherer "conditional" function "conditional_gatherer_rules" took 899ns to process 1 records I0704 22:24:34.837095 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0704 22:24:34.837104 1 gather.go:181] gatherer "conditional" function "remote_configuration" took 1.116µs to process 1 records I0704 22:24:34.837110 1 tasks_processing.go:74] worker 0 stopped. I0704 22:24:34.837173 1 tasks_processing.go:74] worker 2 stopped. I0704 22:24:34.837188 1 gather.go:181] gatherer "conditional" function "rapid_container_logs" took 209.308µs to process 0 records I0704 22:24:34.837211 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=AsExpected message= I0704 22:24:34.837229 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f W0704 22:24:34.862740 1 gather.go:213] can't read cgroups memory usage data: open /sys/fs/cgroup/memory/memory.usage_in_bytes: no such file or directory I0704 22:24:34.862907 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=3b2a7e1fea03cc2613f4250d0abba9a54f94519d9797d54397ec2013971bbb3b I0704 22:24:34.863054 1 diskrecorder.go:70] Writing 137 records to /var/lib/insights-operator/insights-2026-07-04-222434.tar.gz I0704 22:24:34.872968 1 diskrecorder.go:51] Wrote 137 records to disk in 10ms I0704 22:24:34.872998 1 periodic.go:283] Gathering cluster info every 2h0m0s I0704 22:24:34.873025 1 periodic.go:284] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0704 22:25:47.302268 1 diskrecorder.go:170] Found files to send: insights-2026-07-04-222434.tar.gz I0704 22:25:47.302297 1 insightsuploader.go:150] Checking archives to upload periodically every 16m26.410236564s I0704 22:25:47.302306 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0704 22:25:47.320487 1 requests.go:47] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0704 22:25:47.494535 1 requests.go:88] Successfully reported id=2026-07-04T22:25:47Z x-rh-insights-request-id=fb2f958bb6c0497b91c659cb6d6ea094, wrote=76297 I0704 22:25:47.494595 1 insightsuploader.go:187] Uploaded report successfully in 192.280238ms I0704 22:25:47.494617 1 controller.go:119] Initializing last reported time to 2026-07-04T22:25:47Z I0704 22:25:47.494647 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0704 22:25:47.494656 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0704 22:25:47.494663 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0704 22:25:47.501362 1 controller.go:482] The operator is healthy I0704 22:26:17.325764 1 controller.go:482] The operator is healthy I0704 22:26:51.062167 1 insightsreport.go:137] Pulling report from smart-proxy I0704 22:26:51.062192 1 insightsreport.go:149] Retrieving report I0704 22:26:51.075136 1 requests.go:111] Retrieving report for cluster: 2cfd0da7-ea95-4765-aae4-2f77db80a3a5 I0704 22:26:51.075150 1 requests.go:112] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/2cfd0da7-ea95-4765-aae4-2f77db80a3a5/reports I0704 22:26:51.078189 1 requests.go:122] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/2cfd0da7-ea95-4765-aae4-2f77db80a3a5/reports I0704 22:26:51.233944 1 insightsreport.go:184] Report retrieved I0704 22:26:51.279595 1 insightsreport.go:239] Report retrieved correctly I0704 22:28:17.328112 1 controller.go:482] The operator is healthy I0704 22:29:17.300157 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 22:29:17.309239 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:29:17.309259 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 22:29:17.313929 1 secretconfigobserver.go:119] support secret does not exist I0704 22:30:17.332656 1 controller.go:482] The operator is healthy I0704 22:32:17.327815 1 controller.go:482] The operator is healthy I0704 22:34:17.297224 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 22:34:17.301853 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 22:34:17.314881 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 22:34:17.319614 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:34:17.319633 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 22:34:17.327423 1 secretconfigobserver.go:119] support secret does not exist I0704 22:34:17.328476 1 controller.go:482] The operator is healthy I0704 22:36:17.328484 1 controller.go:482] The operator is healthy I0704 22:38:17.328684 1 controller.go:482] The operator is healthy I0704 22:39:17.328424 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 22:39:17.335518 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:39:17.335537 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 22:39:17.339981 1 secretconfigobserver.go:119] support secret does not exist I0704 22:40:17.326181 1 controller.go:482] The operator is healthy