I0702 02:17:05.098484 1 cmd.go:250] Using service-serving-cert provided certificates I0702 02:17:05.098877 1 observer_polling.go:159] Starting file observer W0702 02:17:05.128693 1 builder.go:266] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-6fd96f47fd" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0702 02:17:05.642762 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0702 02:17:05.642999 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0702 02:17:05.643373 1 secure_serving.go:57] Forcing use of http/1.1 only W0702 02:17:05.643394 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0702 02:17:05.643398 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0702 02:17:05.643404 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0702 02:17:05.643407 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0702 02:17:05.643409 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0702 02:17:05.643412 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0702 02:17:05.643443 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0702 02:17:05.648806 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes ChunkSizeMiB CloudDualStackNodeIPs DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy PersistentIPsForVirtualization PrivateHostedZoneAWS SetEIPForNLBIngressController VSphereControlPlaneMachineSet VSphereDriverConfiguration VSphereMultiVCenters VSphereStaticIPs ValidatingAdmissionPolicy AWSClusterHostedDNS AdditionalRoutingCapabilities AutomatedEtcdBackup BootcNodeManagement CSIDriverSharedResource ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ConsolePluginContentSecurityPolicy DNSNameResolver DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example GCPClusterHostedDNS GatewayAPI ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MetricsCollectionProfiles MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NodeSwap NutanixMultiSubnets OVNObservability OnClusterBuild OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SignatureStores SigstoreImageVerification TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0702 02:17:05.648823 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "ChunkSizeMiB", "CloudDualStackNodeIPs", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "SetEIPForNLBIngressController", "VSphereControlPlaneMachineSet", "VSphereDriverConfiguration", "VSphereMultiVCenters", "VSphereStaticIPs", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AdditionalRoutingCapabilities", "AutomatedEtcdBackup", "BootcNodeManagement", "CSIDriverSharedResource", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ConsolePluginContentSecurityPolicy", "DNSNameResolver", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "GCPClusterHostedDNS", "GatewayAPI", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MetricsCollectionProfiles", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OnClusterBuild", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SignatureStores", "SigstoreImageVerification", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0702 02:17:05.651583 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController I0702 02:17:05.651591 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0702 02:17:05.651608 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0702 02:17:05.651606 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0702 02:17:05.651622 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0702 02:17:05.651610 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0702 02:17:05.651874 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0702 02:17:05.652135 1 secure_serving.go:213] Serving securely on [::]:8443 I0702 02:17:05.652171 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" E0702 02:17:05.653570 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18be576c6b03a28b openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"SetEIPForNLBIngressController\", \"VSphereControlPlaneMachineSet\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"VSphereStaticIPs\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AdditionalRoutingCapabilities\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"CSIDriverSharedResource\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ConsolePluginContentSecurityPolicy\", \"DNSNameResolver\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"GCPClusterHostedDNS\", \"GatewayAPI\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MetricsCollectionProfiles\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OnClusterBuild\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SignatureStores\", \"SigstoreImageVerification\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-07-02 02:17:05.648779915 +0000 UTC m=+0.586754306,LastTimestamp:2026-07-02 02:17:05.648779915 +0000 UTC m=+0.586754306,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W0702 02:17:05.655088 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0702 02:17:05.655112 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0702 02:17:05.655210 1 base_controller.go:67] Waiting for caches to sync for ConfigController I0702 02:17:05.661701 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:17:05.661722 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0702 02:17:05.667588 1 secretconfigobserver.go:119] support secret does not exist I0702 02:17:05.673428 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:17:05.674869 1 recorder.go:156] Pruning old reports every 6h53m48s, max age is 288h0m0s I0702 02:17:05.678928 1 secretconfigobserver.go:119] support secret does not exist I0702 02:17:05.681404 1 periodic.go:214] Running clusterconfig gatherer I0702 02:17:05.681412 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0702 02:17:05.681429 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0702 02:17:05.681444 1 tasks_processing.go:45] number of workers: 32 I0702 02:17:05.681476 1 tasks_processing.go:69] worker 31 listening for tasks. I0702 02:17:05.681483 1 tasks_processing.go:71] worker 31 working on image_registries task. I0702 02:17:05.681480 1 tasks_processing.go:69] worker 11 listening for tasks. I0702 02:17:05.681446 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0702 02:17:05.681497 1 insightsreport.go:296] Starting report retriever I0702 02:17:05.681506 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0702 02:17:05.681520 1 tasks_processing.go:69] worker 21 listening for tasks. I0702 02:17:05.681522 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 02:17:05.681531 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 02:17:05.681534 1 tasks_processing.go:69] worker 17 listening for tasks. I0702 02:17:05.681534 1 tasks_processing.go:69] worker 16 listening for tasks. I0702 02:17:05.681541 1 tasks_processing.go:69] worker 2 listening for tasks. I0702 02:17:05.681542 1 tasks_processing.go:69] worker 18 listening for tasks. I0702 02:17:05.681544 1 tasks_processing.go:69] worker 14 listening for tasks. I0702 02:17:05.681549 1 tasks_processing.go:69] worker 19 listening for tasks. I0702 02:17:05.681553 1 tasks_processing.go:69] worker 15 listening for tasks. I0702 02:17:05.681552 1 tasks_processing.go:69] worker 13 listening for tasks. I0702 02:17:05.681558 1 tasks_processing.go:69] worker 20 listening for tasks. I0702 02:17:05.681560 1 tasks_processing.go:69] worker 3 listening for tasks. I0702 02:17:05.681564 1 tasks_processing.go:69] worker 26 listening for tasks. I0702 02:17:05.681569 1 tasks_processing.go:69] worker 7 listening for tasks. I0702 02:17:05.681572 1 tasks_processing.go:69] worker 5 listening for tasks. I0702 02:17:05.681569 1 tasks_processing.go:69] worker 8 listening for tasks. I0702 02:17:05.681571 1 tasks_processing.go:69] worker 12 listening for tasks. I0702 02:17:05.681580 1 tasks_processing.go:69] worker 23 listening for tasks. I0702 02:17:05.681575 1 tasks_processing.go:69] worker 22 listening for tasks. I0702 02:17:05.681584 1 tasks_processing.go:69] worker 4 listening for tasks. I0702 02:17:05.681585 1 tasks_processing.go:69] worker 10 listening for tasks. I0702 02:17:05.681577 1 tasks_processing.go:69] worker 9 listening for tasks. I0702 02:17:05.681591 1 tasks_processing.go:69] worker 25 listening for tasks. I0702 02:17:05.681595 1 tasks_processing.go:69] worker 27 listening for tasks. I0702 02:17:05.681581 1 tasks_processing.go:69] worker 6 listening for tasks. I0702 02:17:05.681587 1 tasks_processing.go:69] worker 24 listening for tasks. I0702 02:17:05.681603 1 tasks_processing.go:69] worker 30 listening for tasks. I0702 02:17:05.681593 1 tasks_processing.go:69] worker 28 listening for tasks. I0702 02:17:05.681594 1 tasks_processing.go:69] worker 29 listening for tasks. I0702 02:17:05.681664 1 tasks_processing.go:71] worker 21 working on ingress task. I0702 02:17:05.681669 1 tasks_processing.go:71] worker 0 working on storage_cluster task. I0702 02:17:05.681674 1 tasks_processing.go:71] worker 8 working on aggregated_monitoring_cr_names task. I0702 02:17:05.681676 1 tasks_processing.go:71] worker 5 working on cluster_apiserver task. I0702 02:17:05.681687 1 tasks_processing.go:71] worker 12 working on tsdb_status task. I0702 02:17:05.681697 1 tasks_processing.go:71] worker 6 working on openshift_machine_api_events task. I0702 02:17:05.681740 1 tasks_processing.go:71] worker 11 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0702 02:17:05.681784 1 tasks_processing.go:71] worker 13 working on crds task. I0702 02:17:05.681683 1 tasks_processing.go:71] worker 9 working on openstack_version task. I0702 02:17:05.681668 1 tasks_processing.go:71] worker 29 working on monitoring_persistent_volumes task. I0702 02:17:05.681756 1 tasks_processing.go:71] worker 24 working on machines task. I0702 02:17:05.681757 1 tasks_processing.go:71] worker 17 working on oauths task. I0702 02:17:05.681759 1 tasks_processing.go:71] worker 25 working on openshift_logging task. I0702 02:17:05.681761 1 tasks_processing.go:71] worker 30 working on mutating_webhook_configurations task. I0702 02:17:05.681762 1 tasks_processing.go:71] worker 16 working on operators task. I0702 02:17:05.681763 1 tasks_processing.go:71] worker 27 working on schedulers task. I0702 02:17:05.681766 1 tasks_processing.go:71] worker 2 working on container_images task. I0702 02:17:05.681765 1 tasks_processing.go:71] worker 28 working on service_accounts task. I0702 02:17:05.681770 1 tasks_processing.go:71] worker 22 working on olm_operators task. I0702 02:17:05.681771 1 tasks_processing.go:71] worker 18 working on silenced_alerts task. I0702 02:17:05.681770 1 tasks_processing.go:71] worker 4 working on support_secret task. I0702 02:17:05.681775 1 tasks_processing.go:71] worker 20 working on container_runtime_configs task. I0702 02:17:05.681775 1 tasks_processing.go:71] worker 14 working on image task. I0702 02:17:05.681774 1 tasks_processing.go:71] worker 23 working on nodenetworkstates task. I0702 02:17:05.681778 1 tasks_processing.go:71] worker 19 working on ceph_cluster task. I0702 02:17:05.681780 1 tasks_processing.go:71] worker 3 working on pod_network_connectivity_checks task. I0702 02:17:05.681781 1 tasks_processing.go:71] worker 15 working on certificate_signing_requests task. I0702 02:17:05.681780 1 tasks_processing.go:71] worker 10 working on sap_datahubs task. I0702 02:17:05.681784 1 tasks_processing.go:71] worker 26 working on pdbs task. I0702 02:17:05.681787 1 tasks_processing.go:71] worker 7 working on nodes task. I0702 02:17:05.681665 1 tasks_processing.go:71] worker 1 working on version task. I0702 02:17:05.686191 1 tasks_processing.go:71] worker 0 working on metrics task. I0702 02:17:05.686207 1 gather.go:181] gatherer "clusterconfig" function "storage_cluster" took 4.508262ms to process 0 records E0702 02:17:05.687064 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:36870->172.30.0.10:53: read: connection refused I0702 02:17:05.687096 1 tasks_processing.go:71] worker 0 working on openstack_dataplanenodesets task. E0702 02:17:05.687076 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:40754->172.30.0.10:53: read: connection refused E0702 02:17:05.687076 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:36870->172.30.0.10:53: read: connection refused E0702 02:17:05.687174 1 gather.go:144] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:36870->172.30.0.10:53: read: connection refused I0702 02:17:05.687203 1 gather.go:181] gatherer "clusterconfig" function "metrics" took 892.786µs to process 0 records E0702 02:17:05.687221 1 gather.go:144] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:40754->172.30.0.10:53: read: connection refused I0702 02:17:05.687232 1 tasks_processing.go:71] worker 18 working on sap_pods task. I0702 02:17:05.687235 1 gather.go:181] gatherer "clusterconfig" function "silenced_alerts" took 4.390856ms to process 0 records E0702 02:17:05.687248 1 gather.go:144] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:36870->172.30.0.10:53: read: connection refused I0702 02:17:05.687263 1 gather.go:181] gatherer "clusterconfig" function "tsdb_status" took 5.474337ms to process 0 records I0702 02:17:05.687298 1 tasks_processing.go:71] worker 12 working on machine_config_pools task. I0702 02:17:05.688489 1 tasks_processing.go:71] worker 9 working on config_maps task. I0702 02:17:05.688524 1 gather.go:181] gatherer "clusterconfig" function "openstack_version" took 6.602144ms to process 0 records I0702 02:17:05.691240 1 tasks_processing.go:71] worker 24 working on infrastructures task. E0702 02:17:05.691248 1 gather.go:144] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0702 02:17:05.691260 1 gather.go:181] gatherer "clusterconfig" function "machines" took 9.085005ms to process 0 records I0702 02:17:05.696027 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0702 02:17:05.696044 1 controller.go:317] The initial operator extension status is healthy I0702 02:17:05.696051 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0702 02:17:05.696056 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0702 02:17:05.696059 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0702 02:17:05.696075 1 controller.go:457] The operator is still being initialized I0702 02:17:05.696083 1 controller.go:482] The operator is healthy I0702 02:17:05.696150 1 sca.go:98] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/certificates. Next check is in 8h0m0s I0702 02:17:05.696214 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0702 02:17:05.696548 1 base_controller.go:67] Waiting for caches to sync for LoggingSyncer W0702 02:17:05.696557 1 operator.go:288] started I0702 02:17:05.699062 1 tasks_processing.go:71] worker 25 working on lokistack task. I0702 02:17:05.699116 1 gather.go:181] gatherer "clusterconfig" function "openshift_logging" took 16.674389ms to process 0 records I0702 02:17:05.699323 1 tasks_processing.go:71] worker 6 working on openstack_dataplanedeployments task. I0702 02:17:05.699398 1 gather.go:181] gatherer "clusterconfig" function "openshift_machine_api_events" took 17.616971ms to process 0 records I0702 02:17:05.699472 1 tasks_processing.go:71] worker 5 working on jaegers task. I0702 02:17:05.699793 1 recorder.go:70] Recording config/apiserver with fingerprint=484ab18dcf4c7a04cd5e204f8b273dcfe2404382adb1d1c6325d94c8ef044b51 I0702 02:17:05.699814 1 gather.go:181] gatherer "clusterconfig" function "cluster_apiserver" took 17.778162ms to process 1 records I0702 02:17:05.699828 1 gather.go:181] gatherer "clusterconfig" function "container_runtime_configs" took 16.492071ms to process 0 records I0702 02:17:05.699858 1 gather.go:181] gatherer "clusterconfig" function "ceph_cluster" took 16.230025ms to process 0 records I0702 02:17:05.699893 1 tasks_processing.go:71] worker 20 working on openstack_controlplanes task. I0702 02:17:05.699933 1 tasks_processing.go:71] worker 19 working on authentication task. I0702 02:17:05.699990 1 tasks_processing.go:71] worker 17 working on proxies task. I0702 02:17:05.700165 1 recorder.go:70] Recording config/oauth with fingerprint=009b1fb7d6ab82de916e3425b5fc8625cc96f8567f40725d4cee98331ec42955 I0702 02:17:05.700181 1 gather.go:181] gatherer "clusterconfig" function "oauths" took 17.403921ms to process 1 records I0702 02:17:05.700265 1 tasks_processing.go:71] worker 21 working on feature_gates task. I0702 02:17:05.700410 1 recorder.go:70] Recording config/ingress with fingerprint=962d33e86628e856a3451545d04a7ec4772b9c5d6143fde1bd872afcacfb494d I0702 02:17:05.700425 1 gather.go:181] gatherer "clusterconfig" function "ingress" took 18.115653ms to process 1 records I0702 02:17:05.700438 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkstates" took 16.622236ms to process 0 records E0702 02:17:05.700446 1 gather.go:144] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0702 02:17:05.700453 1 tasks_processing.go:71] worker 23 working on networks task. I0702 02:17:05.700455 1 gather.go:181] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 16.424568ms to process 0 records I0702 02:17:05.700464 1 gather.go:181] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 17.947182ms to process 0 records I0702 02:17:05.700547 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=99297b76452a4c97f8a653f2d4fe14a2556db56cd7f201ae1c8fe62f80db048f I0702 02:17:05.700560 1 gather.go:181] gatherer "clusterconfig" function "schedulers" took 17.499718ms to process 1 records I0702 02:17:05.700618 1 tasks_processing.go:71] worker 27 working on cost_management_metrics_configs task. I0702 02:17:05.700628 1 tasks_processing.go:71] worker 29 working on machine_sets task. I0702 02:17:05.700664 1 recorder.go:70] Recording config/image with fingerprint=775675a07292686bf7b5c1aedeac55dee771eafb9afc377c71af98cb0a596c7b I0702 02:17:05.700679 1 gather.go:181] gatherer "clusterconfig" function "image" took 17.405703ms to process 1 records I0702 02:17:05.700688 1 tasks_processing.go:71] worker 14 working on sap_config task. I0702 02:17:05.700620 1 tasks_processing.go:71] worker 3 working on storage_classes task. I0702 02:17:05.705228 1 tasks_processing.go:71] worker 10 working on dvo_metrics task. I0702 02:17:05.705239 1 gather.go:181] gatherer "clusterconfig" function "sap_datahubs" took 21.670415ms to process 0 records I0702 02:17:05.705631 1 tasks_processing.go:71] worker 0 working on overlapping_namespace_uids task. I0702 02:17:05.705641 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 18.524213ms to process 0 records I0702 02:17:05.706100 1 tasks_processing.go:71] worker 12 working on install_plans task. I0702 02:17:05.706112 1 gather.go:181] gatherer "clusterconfig" function "machine_config_pools" took 18.784706ms to process 0 records I0702 02:17:05.716496 1 tasks_processing.go:71] worker 18 working on nodenetworkconfigurationpolicies task. I0702 02:17:05.716528 1 gather.go:181] gatherer "clusterconfig" function "sap_pods" took 29.25411ms to process 0 records E0702 02:17:05.716560 1 gather.go:144] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0702 02:17:05.716574 1 gather.go:181] gatherer "clusterconfig" function "support_secret" took 33.695694ms to process 0 records I0702 02:17:05.716617 1 tasks_processing.go:71] worker 4 working on machine_healthchecks task. I0702 02:17:05.717414 1 tasks_processing.go:71] worker 31 working on operators_pods_and_events task. I0702 02:17:05.718062 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=4963789101009c6d19dfb17d4f3e6d4b407125314f77dd0486bd9c86b5db775d I0702 02:17:05.718077 1 gather.go:181] gatherer "clusterconfig" function "image_registries" took 35.924286ms to process 1 records I0702 02:17:05.718554 1 tasks_processing.go:71] worker 30 working on ingress_certificates task. I0702 02:17:05.718755 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=a14d77a5a931ecdcdad604c0b67e8b0c380d9b03f439cadc383d83b9f9477bda I0702 02:17:05.718802 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=c55bff6c585f1244ea107ee4d0b72c53f006f559abed91f168cea887a1781f58 I0702 02:17:05.718863 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=6e513384a7343f53d60e2f8a560ff6f23598947ffd012fac943580bfc33596f6 I0702 02:17:05.718880 1 gather.go:181] gatherer "clusterconfig" function "mutating_webhook_configurations" took 36.152403ms to process 3 records I0702 02:17:05.719500 1 tasks_processing.go:71] worker 24 working on image_pruners task. I0702 02:17:05.719792 1 controller.go:203] Source scaController *sca.Controller is not ready I0702 02:17:05.719877 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0702 02:17:05.719945 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0702 02:17:05.719964 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0702 02:17:05.719973 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0702 02:17:05.720184 1 controller.go:457] The operator is still being initialized I0702 02:17:05.720199 1 controller.go:482] The operator is healthy I0702 02:17:05.721746 1 recorder.go:70] Recording config/infrastructure with fingerprint=0e900a31f8ac1762a02b86cf6d464ae5a8bd913ba96ea480bf1216575488f37b I0702 02:17:05.721814 1 gather.go:181] gatherer "clusterconfig" function "infrastructures" took 28.244773ms to process 1 records I0702 02:17:05.722005 1 recorder.go:70] Recording config/olm_operators with fingerprint=292c1d3095e399a2fb42a0722cac20f1a599323b3802f013bfad4ea049c3b0be I0702 02:17:05.722048 1 gather.go:181] gatherer "clusterconfig" function "olm_operators" took 38.529694ms to process 1 records I0702 02:17:05.722078 1 gather.go:181] gatherer "clusterconfig" function "jaegers" took 21.908191ms to process 0 records I0702 02:17:05.722114 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 22.108921ms to process 0 records I0702 02:17:05.722145 1 gather.go:181] gatherer "clusterconfig" function "lokistack" took 22.456961ms to process 0 records I0702 02:17:05.722156 1 gather.go:181] gatherer "clusterconfig" function "cost_management_metrics_configs" took 20.95374ms to process 0 records I0702 02:17:05.722164 1 gather.go:181] gatherer "clusterconfig" function "openstack_controlplanes" took 21.689571ms to process 0 records I0702 02:17:05.722175 1 gather.go:181] gatherer "clusterconfig" function "machine_sets" took 20.984538ms to process 0 records I0702 02:17:05.722181 1 gather.go:181] gatherer "clusterconfig" function "sap_config" took 21.096734ms to process 0 records I0702 02:17:05.722029 1 tasks_processing.go:71] worker 22 working on validating_webhook_configurations task. I0702 02:17:05.722318 1 tasks_processing.go:74] worker 27 stopped. I0702 02:17:05.722334 1 tasks_processing.go:71] worker 14 working on machine_configs task. I0702 02:17:05.722375 1 tasks_processing.go:71] worker 6 working on active_alerts task. I0702 02:17:05.722799 1 tasks_processing.go:71] worker 5 working on machine_autoscalers task. I0702 02:17:05.723044 1 tasks_processing.go:71] worker 25 working on clusterroles task. I0702 02:17:05.723086 1 tasks_processing.go:71] worker 20 working on node_logs task. I0702 02:17:05.723736 1 tasks_processing.go:74] worker 29 stopped. I0702 02:17:05.723966 1 tasks_processing.go:74] worker 19 stopped. I0702 02:17:05.725715 1 recorder.go:70] Recording config/authentication with fingerprint=a6515266722a58ef325c41881f4c88f91ef71b4fb03cd84e6799ffa72de5f22e I0702 02:17:05.725736 1 gather.go:181] gatherer "clusterconfig" function "authentication" took 24.016162ms to process 1 records I0702 02:17:05.725812 1 recorder.go:70] Recording config/proxy with fingerprint=a9640c502f96a728ada348c93315e0c26b21168d623bc940b7e7cf4a5788094e I0702 02:17:05.725827 1 gather.go:181] gatherer "clusterconfig" function "proxies" took 24.757726ms to process 1 records I0702 02:17:05.725859 1 tasks_processing.go:74] worker 17 stopped. I0702 02:17:05.725998 1 tasks_processing.go:74] worker 7 stopped. I0702 02:17:05.728043 1 recorder.go:70] Recording config/node/ip-10-0-0-129.ec2.internal with fingerprint=f62f138f530c37dd1555abac55222e8d53c114f8fc3356a12f5ff7d1fca80fde I0702 02:17:05.728230 1 recorder.go:70] Recording config/node/ip-10-0-1-85.ec2.internal with fingerprint=d25e37a77f3554748102c6273088f4b79ed80008ab89f5249f41363921c5eace I0702 02:17:05.728352 1 recorder.go:70] Recording config/node/ip-10-0-2-19.ec2.internal with fingerprint=cdcff5674bd199125fcace17fb426c91b09ca80300ecb45a4add1f61f990d9ca I0702 02:17:05.728386 1 gather.go:181] gatherer "clusterconfig" function "nodes" took 42.352247ms to process 3 records E0702 02:17:05.728419 1 gather.go:144] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0702 02:17:05.728449 1 gather.go:181] gatherer "clusterconfig" function "machine_healthchecks" took 9.639535ms to process 0 records I0702 02:17:05.728704 1 recorder.go:70] Recording config/network with fingerprint=fd38d0de4736ba5c051aa47f9f2bcaa771311b1a78dd59b417cdae6af5889950 I0702 02:17:05.728758 1 gather.go:181] gatherer "clusterconfig" function "networks" took 26.890571ms to process 1 records I0702 02:17:05.728913 1 recorder.go:70] Recording config/pdbs/openshift-console/console with fingerprint=9fd9899d04f546c906cd6e80faf0131253c45d8af5dfa9d0e4d2088b25910fb5 I0702 02:17:05.728976 1 recorder.go:70] Recording config/pdbs/openshift-console/downloads with fingerprint=cfdd22c6b0dacf238ce5ab6d55d7fafed97100e33a1def94186a64ede625dbad I0702 02:17:05.729022 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=9c34a6ffe39e83a3134fe52aefd40f89f0c46b29f8c211a5c6b88761e6048b3a I0702 02:17:05.729067 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=0bb9cc6b7660f63f546510ca217cf2913e40d3ca8c39d4e9c2eb1f84db07638e I0702 02:17:05.729121 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=f5a7a6838a01426329dc72ef461b166a4200ae30970f9a9203dac92dd8043eb7 I0702 02:17:05.729167 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=66515c6adc89dc13c6bf857fa21ed7e5f2cd64f3830c88c5b3801131839b3dd5 I0702 02:17:05.729225 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=092c7e29367c7348f2bcafc804889382c7ed5429eba22bbb5efe7ed2f6b185ac I0702 02:17:05.729280 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=afb712d4687454c62a249232bff7b62bd74a3779598c41f61874bca45e9c4729 I0702 02:17:05.729329 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=609969100b263d8388497421757629e60294553680de6264aab3c9c429cf2160 I0702 02:17:05.729379 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=560af3e98474a1bd180ec015bb746a0e9947baee270588061250bdd7b596712f I0702 02:17:05.729417 1 gather.go:181] gatherer "clusterconfig" function "pdbs" took 44.326038ms to process 10 records I0702 02:17:05.729538 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=06ffb57d87febf40ffd70f0a4817d86798dd706e9701f4a59b8cefacc343b1a1 I0702 02:17:05.729596 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=8955e3b29befc2b904607a313f0be6a1ec1ba08c27cfc0e2fb8f51a61a4fed80 I0702 02:17:05.729637 1 gather.go:181] gatherer "clusterconfig" function "storage_classes" took 28.18757ms to process 2 records I0702 02:17:05.729678 1 tasks_processing.go:74] worker 3 stopped. I0702 02:17:05.729718 1 tasks_processing.go:74] worker 4 stopped. I0702 02:17:05.729873 1 tasks_processing.go:74] worker 23 stopped. I0702 02:17:05.729883 1 tasks_processing.go:74] worker 26 stopped. I0702 02:17:05.731537 1 tasks_processing.go:74] worker 21 stopped. I0702 02:17:05.731862 1 recorder.go:70] Recording config/featuregate with fingerprint=8ba946c480be68346bb41866fb2d4d1c09af912714cd835ef59e4eb8ef848b92 I0702 02:17:05.731956 1 gather.go:181] gatherer "clusterconfig" function "feature_gates" took 31.257333ms to process 1 records I0702 02:17:05.732001 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0702 02:17:05.732035 1 gather.go:181] gatherer "clusterconfig" function "overlapping_namespace_uids" took 26.048194ms to process 1 records I0702 02:17:05.732079 1 tasks_processing.go:74] worker 0 stopped. I0702 02:17:05.732336 1 tasks_processing.go:74] worker 18 stopped. I0702 02:17:05.732395 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 15.826001ms to process 0 records I0702 02:17:05.732526 1 tasks_processing.go:74] worker 15 stopped. I0702 02:17:05.732602 1 gather.go:181] gatherer "clusterconfig" function "certificate_signing_requests" took 49.08431ms to process 0 records I0702 02:17:05.733490 1 tasks_processing.go:74] worker 13 stopped. I0702 02:17:05.734682 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=95cabe684ddb0f98c59b6cf8b675cd6f7e08daf6090419b80c729db23685d926 I0702 02:17:05.735188 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=41955cd77a66750d6a56d59518cd02e4d383057b1ceb80e1d37ada5ffa9e7dd1 I0702 02:17:05.735244 1 gather.go:181] gatherer "clusterconfig" function "crds" took 51.684895ms to process 2 records E0702 02:17:05.735375 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:60741->172.30.0.10:53: read: connection refused I0702 02:17:05.735469 1 tasks_processing.go:74] worker 6 stopped. E0702 02:17:05.735507 1 gather.go:144] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.9:60741->172.30.0.10:53: read: connection refused I0702 02:17:05.735595 1 gather.go:181] gatherer "clusterconfig" function "active_alerts" took 13.046484ms to process 0 records I0702 02:17:05.736792 1 tasks_processing.go:74] worker 5 stopped. I0702 02:17:05.736875 1 gather.go:181] gatherer "clusterconfig" function "machine_autoscalers" took 13.928449ms to process 0 records I0702 02:17:05.737349 1 tasks_processing.go:74] worker 24 stopped. I0702 02:17:05.737733 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=5073713836f894715ec0d49b45af772089f9a19c661566361ba328511fb584e8 I0702 02:17:05.737790 1 gather.go:181] gatherer "clusterconfig" function "image_pruners" took 17.706403ms to process 1 records W0702 02:17:05.738686 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:17:05.738884 1 tasks_processing.go:74] worker 1 stopped. I0702 02:17:05.739348 1 requests.go:204] Asking for SCA certificate for x86_64 architecture I0702 02:17:05.739624 1 recorder.go:70] Recording config/version with fingerprint=d1aae745093a92726b8de54a72bccdaf1e874c57826c1d4d0e08e861f38c2429 I0702 02:17:05.739682 1 recorder.go:70] Recording config/id with fingerprint=1582d8c1dd37431dcbc900ee7dceeac3f4aaa7e9ac3da920f00d5e5e05f93c4e I0702 02:17:05.739716 1 gather.go:181] gatherer "clusterconfig" function "version" took 55.187912ms to process 2 records I0702 02:17:05.740086 1 tasks_processing.go:74] worker 20 stopped. I0702 02:17:05.740147 1 gather.go:181] gatherer "clusterconfig" function "node_logs" took 16.970424ms to process 0 records I0702 02:17:05.740413 1 tasks_processing.go:74] worker 22 stopped. I0702 02:17:05.740545 1 recorder.go:70] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=3a9fd4c31945d0178b9d55ac9cb494d5ba3cefa0a2ee65d71fae4d22a88bf684 I0702 02:17:05.740634 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=6e62a62b646a438c688eab2aab4caa24732796830beaf926d694b4c6e10306d8 I0702 02:17:05.740749 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=37219716383d8fc27efd11ecb2799740f94cedf463ddf950483c60ac9732a36f I0702 02:17:05.740788 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=c0e9affb6844d2dca93061ff9b35bbe34c3e5e3fce298e7febd51e1451dd90e7 I0702 02:17:05.740823 1 recorder.go:70] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=7a704180e3bf636c0ddfcb11f180403f46ddb03030a63437b71b2e1cbc1615e0 I0702 02:17:05.740897 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=ded72019b77979fddd3e7073984a8844e76a2c9cb775ba6d9ad5357ead666a4a I0702 02:17:05.740935 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=1b5d4d33fd006ec0d29469401f791fa2b48f238a6235ea81b6c8fcd203843c00 I0702 02:17:05.740972 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=af77b79d5657f2940b7689f6f963e711dcd7664a5c0d3c386dcbffe02d7f7d4e I0702 02:17:05.741027 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=657318478e0dce1b631a95a172f3d6c5dbce55735126e01fda1adf0b0e3ae7ca I0702 02:17:05.741103 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=9fe5d67d272d38e44a2aa6061a5a32c5a75674b1391051097f4b6a053b9fc0f9 I0702 02:17:05.741143 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=5653f6badd4643aa9f121f468e1d27ab315b1a32cacaeb59ac8c09957abbb935 I0702 02:17:05.741194 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=f2dc974d30683e91479938f18f5cdb3bd0ee9b15f79beff02165a94a5e14d448 I0702 02:17:05.741266 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=3e13a91423d9e5738bdcd13dadd1f820c7cbfc098588a112f71feefc37f392c1 I0702 02:17:05.741291 1 gather.go:181] gatherer "clusterconfig" function "validating_webhook_configurations" took 18.216806ms to process 13 records E0702 02:17:05.741316 1 gather.go:144] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0702 02:17:05.741389 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0702 02:17:05.741404 1 gather.go:181] gatherer "clusterconfig" function "machine_configs" took 18.144047ms to process 1 records I0702 02:17:05.741411 1 gather.go:181] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 59.578024ms to process 0 records I0702 02:17:05.741417 1 tasks_processing.go:74] worker 8 stopped. I0702 02:17:05.741391 1 tasks_processing.go:74] worker 14 stopped. I0702 02:17:05.743291 1 tasks_processing.go:74] worker 25 stopped. I0702 02:17:05.743599 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=dafcb3aed0762560bc1038b341ac5b4bc2dceb6c08a3a8d8a7c8c0c4ae6a09b3 I0702 02:17:05.743745 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=7335960d96a2d745fd88dcc81db4783f3f8c8def3bbfd2834e019c2875fe9f75 I0702 02:17:05.743758 1 gather.go:181] gatherer "clusterconfig" function "clusterroles" took 20.190893ms to process 2 records W0702 02:17:05.744279 1 sca.go:117] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:56178->172.30.0.10:53: read: connection refused I0702 02:17:05.744338 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:56178->172.30.0.10:53: read: connection refused E0702 02:17:05.744420 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%!a(MISSING)f3aa7e-1984-4c38-b1ca-712be8222f0b%!+(MISSING)and+status+is+%!a(MISSING)ccepted%!"(MISSING): dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:56178->172.30.0.10:53: read: connection refused I0702 02:17:05.744454 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%278af3aa7e-1984-4c38-b1ca-712be8222f0b%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:56178->172.30.0.10:53: read: connection refused I0702 02:17:05.752390 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0702 02:17:05.752396 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0702 02:17:05.752413 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0702 02:17:05.753084 1 tasks_processing.go:74] worker 2 stopped. I0702 02:17:05.754580 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl with fingerprint=f35548f5b3308c8cc347ded739b8ccf825abe9d824cf4ad3aa736b3829bbdd78 I0702 02:17:05.754683 1 recorder.go:70] Recording config/running_containers with fingerprint=e619ecc57be3b22963910b0390a8cc748559a3bea33d274f085bec00e2988378 I0702 02:17:05.754696 1 gather.go:181] gatherer "clusterconfig" function "container_images" took 70.459465ms to process 2 records I0702 02:17:05.754707 1 gather.go:181] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 72.378561ms to process 0 records I0702 02:17:05.754733 1 tasks_processing.go:74] worker 11 stopped. I0702 02:17:05.756229 1 base_controller.go:73] Caches are synced for ConfigController I0702 02:17:05.756242 1 base_controller.go:110] Starting #1 worker of ConfigController controller ... I0702 02:17:05.760537 1 tasks_processing.go:74] worker 9 stopped. E0702 02:17:05.760550 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0702 02:17:05.760555 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0702 02:17:05.760559 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0702 02:17:05.760591 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0702 02:17:05.760613 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0702 02:17:05.760621 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=e1a0ad594db3a5d5d50e9e25d7b235af20f84f5def41537f127febe525b834f2 I0702 02:17:05.760634 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0702 02:17:05.760683 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0702 02:17:05.760690 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0702 02:17:05.760695 1 gather.go:181] gatherer "clusterconfig" function "config_maps" took 72.029764ms to process 6 records I0702 02:17:05.764465 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:17:05.764974 1 tasks_processing.go:74] worker 30 stopped. E0702 02:17:05.764986 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0702 02:17:05.764991 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2r9ndq7b0fnq5g5hhp700fgmjc9j5kpc-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2r9ndq7b0fnq5g5hhp700fgmjc9j5kpc-primary-cert-bundle-secret" not found I0702 02:17:05.765050 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=f9473d8f0a0fa716c13248b0605747fb429f301575c0bb5145e5ac8add5038bb I0702 02:17:05.765061 1 gather.go:181] gatherer "clusterconfig" function "ingress_certificates" took 46.403203ms to process 1 records I0702 02:17:05.796848 1 base_controller.go:73] Caches are synced for LoggingSyncer I0702 02:17:05.796861 1 base_controller.go:110] Starting #1 worker of LoggingSyncer controller ... I0702 02:17:05.809634 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress-operator" I0702 02:17:06.356532 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0702 02:17:06.556788 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" W0702 02:17:06.730163 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:17:06.952243 1 gather_cluster_operator_pods_and_events.go:121] Found 25 pods with 28 containers I0702 02:17:06.952258 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 898779 bytes I0702 02:17:06.952778 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-7d4ddc79d6-n6vxl pod in namespace openshift-console-operator (previous: true). I0702 02:17:07.183501 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-7d4ddc79d6-n6vxl pod in namespace openshift-console-operator (previous: false). I0702 02:17:07.359394 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0702 02:17:07.364976 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-76d7c69695-5h574 pod in namespace openshift-console (previous: false). I0702 02:17:07.579305 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-76d7c69695-dslh6 pod in namespace openshift-console (previous: false). W0702 02:17:07.730641 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:17:07.752696 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:07.752710 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-966565477-c8k29 pod in namespace openshift-console (previous: false). I0702 02:17:07.960016 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-966565477-cdzh2 pod in namespace openshift-console (previous: false). I0702 02:17:07.960706 1 tasks_processing.go:74] worker 16 stopped. I0702 02:17:07.960891 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=03e9783647b3d1edd0ebfa888aaaa108bda243c0c105b09da0821aa86f68df91 I0702 02:17:07.960946 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=183539bbe89c989a326011f124cb26e91d4dc0406689baecde44742b1c4a8e80 I0702 02:17:07.960972 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=3180cb9382739cdb8c9f9afe5eb656bb2d37129e8013eaec0fd153878a128bdd I0702 02:17:07.960983 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0702 02:17:07.961005 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=43e081d0e272de1123c9d447e407b17f9def7d1f54107116fead2ffeb8fb3f4e I0702 02:17:07.961023 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0702 02:17:07.961047 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=f338d871df44e4f5466780741f04f393a1bb946c5b3576e8df343457e1d2443c I0702 02:17:07.961074 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=c22605d2a1bd305acf386dad68702660c827543830c9daf74b1037e615b0070a I0702 02:17:07.961099 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=5d867e73190518a9da982aa77ca7f287a16ad022cb395a0c75f2d093d0ac89c2 I0702 02:17:07.961141 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=59d005c346ab31b43d977531d50485707d0818b1e424c60209b294551c54c2b8 I0702 02:17:07.961612 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=9dc6416641c719d28fea4d5237ff3eec7456d2d89bf69eedb14c0bf6a6f5f555 I0702 02:17:07.961796 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=0d979125bee6706163565a81a73c13f885a6fa0bea24d58edfaaec9b729bcd49 I0702 02:17:07.961909 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0702 02:17:07.961975 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=09d725d2e75803634eca618be320a4ab46a24e0e30f60909266be5de4fb2a15d I0702 02:17:07.962027 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0702 02:17:07.962085 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=2238f858bd48b493fccf0e453c32e52c67c17d2fc8691a0f420ea20a28ca9d84 I0702 02:17:07.962120 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0702 02:17:07.962186 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=d264e6f3dc3fa7dae3363dc65a6b849b6f32313f59570448dee931037c8aba2f I0702 02:17:07.962222 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0702 02:17:07.962287 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=365aea7a90921c64a6b682170e3ee597547b27bb728e3895a2ca3d9dd69d0095 I0702 02:17:07.962482 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=63b69307959e4a2d00cedbc3304e83d1f20c51fcabc2513337e7446066536cd9 I0702 02:17:07.962570 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0702 02:17:07.962626 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=4fd727fbd5c690f5b548d99de5e62a088a0856363bf1d72dd8bd8ee359e4cd46 I0702 02:17:07.962696 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=a5a51c3d2c4faa0124fad6b4de165eee467ec8974fd1e4760685c6d162f49dc6 I0702 02:17:07.962731 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0702 02:17:07.962781 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=800466b169778f7fd474f4403c4309434426b2ee84284fbbc7378899685b868a I0702 02:17:07.962822 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0702 02:17:07.962896 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=e17a6cb2be9f6dac5c982888ee06354cb65a8aae36f73e8643f8c6ae5bab0d12 I0702 02:17:07.962942 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=ae10eb5f4c97710296fb3e01c35648df3389fa31c31a82f08f368c41d7900219 I0702 02:17:07.962997 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=8999419466465b04a6ed4b5cec338159a07541483e3b258432c20d705076b089 I0702 02:17:07.963042 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=f46c12706f2ae394367dcd5405542110f627d9216725590875ad957612d839c0 I0702 02:17:07.963101 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=74000b9904c44f628a7ba95d7b59fe4990ed33f054d88050173ce36b6d18f0f2 I0702 02:17:07.963147 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0702 02:17:07.963215 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=2fa8ec9a358477df4c969320b7c9d73fe892f4c0d2a60f277128e5c545915036 I0702 02:17:07.963272 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0702 02:17:07.963304 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0702 02:17:07.963334 1 gather.go:181] gatherer "clusterconfig" function "operators" took 2.2782704s to process 36 records I0702 02:17:08.180096 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-6d4c9b4b6c-jwv2l pod in namespace openshift-console (previous: false). I0702 02:17:08.360106 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:08.360125 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-6d4c9b4b6c-lgpqk pod in namespace openshift-console (previous: false). I0702 02:17:08.573412 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:08.573431 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-d9n2t pod in namespace openshift-dns (previous: false). W0702 02:17:08.730900 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:17:08.759527 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-d9n2t pod in namespace openshift-dns (previous: false). I0702 02:17:08.959615 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-n9jnm pod in namespace openshift-dns (previous: false). I0702 02:17:09.157712 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-n9jnm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-n9jnm\" is waiting to start: ContainerCreating" I0702 02:17:09.157729 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-n9jnm\" is waiting to start: ContainerCreating" I0702 02:17:09.157739 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-n9jnm pod in namespace openshift-dns (previous: false). I0702 02:17:09.358625 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-n9jnm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-n9jnm\" is waiting to start: ContainerCreating" I0702 02:17:09.358642 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-n9jnm\" is waiting to start: ContainerCreating" I0702 02:17:09.358670 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-sw4fz pod in namespace openshift-dns (previous: false). I0702 02:17:09.559230 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-sw4fz pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-sw4fz\" is waiting to start: ContainerCreating" I0702 02:17:09.559247 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-sw4fz\" is waiting to start: ContainerCreating" I0702 02:17:09.559256 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-sw4fz pod in namespace openshift-dns (previous: false). W0702 02:17:09.731263 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:17:09.760404 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-sw4fz pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-sw4fz\" is waiting to start: ContainerCreating" I0702 02:17:09.760422 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-sw4fz\" is waiting to start: ContainerCreating" I0702 02:17:09.760432 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-djlnp pod in namespace openshift-dns (previous: false). I0702 02:17:09.963239 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:09.963264 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-dr9ft pod in namespace openshift-dns (previous: false). I0702 02:17:10.159034 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-qwjjs pod in namespace openshift-dns (previous: false). I0702 02:17:10.360121 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:10.360177 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-64d864fc7f-b9lks pod in namespace openshift-image-registry (previous: false). I0702 02:17:10.559816 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-64d864fc7f-b9lks pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-64d864fc7f-b9lks\" is waiting to start: ContainerCreating" I0702 02:17:10.559851 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-64d864fc7f-b9lks\" is waiting to start: ContainerCreating" I0702 02:17:10.559865 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6d74c9594-6ff7b pod in namespace openshift-image-registry (previous: false). I0702 02:17:10.736868 1 tasks_processing.go:74] worker 10 stopped. I0702 02:17:10.736888 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0702 02:17:10.736900 1 gather.go:181] gatherer "clusterconfig" function "dvo_metrics" took 5.03162183s to process 1 records I0702 02:17:10.760540 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6d74c9594-9449v pod in namespace openshift-image-registry (previous: false). I0702 02:17:10.962546 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-gj2q8 pod in namespace openshift-image-registry (previous: false). I0702 02:17:11.170217 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-mp67w pod in namespace openshift-image-registry (previous: false). I0702 02:17:11.358620 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:11.358636 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-wkvbx pod in namespace openshift-image-registry (previous: false). I0702 02:17:11.559256 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:17:11.559273 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-655f54c855-522sw pod in namespace openshift-ingress (previous: false). I0702 02:17:11.760372 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-848bc4b65b-6csqh pod in namespace openshift-ingress (previous: false). I0702 02:17:11.958889 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-848bc4b65b-hvvmq pod in namespace openshift-ingress (previous: false). I0702 02:17:12.160612 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-927fr pod in namespace openshift-ingress-canary (previous: false). I0702 02:17:12.359846 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-927fr pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-927fr\" is waiting to start: ContainerCreating" I0702 02:17:12.359865 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-927fr\" is waiting to start: ContainerCreating" I0702 02:17:12.359897 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-hjjfs pod in namespace openshift-ingress-canary (previous: false). I0702 02:17:12.558808 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-hjjfs pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-hjjfs\" is waiting to start: ContainerCreating" I0702 02:17:12.558824 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-hjjfs\" is waiting to start: ContainerCreating" I0702 02:17:12.558855 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-vdnpm pod in namespace openshift-ingress-canary (previous: false). I0702 02:17:12.758979 1 tasks_processing.go:74] worker 31 stopped. I0702 02:17:12.759216 1 recorder.go:70] Recording events/openshift-console-operator with fingerprint=19417e912e9fc782e14e6f058dc0227abf8fff4b0defd467d853e1fc840964c0 I0702 02:17:12.759302 1 recorder.go:70] Recording events/openshift-console with fingerprint=231441229edbd44f43a230d9c35faa3af65760511947304d845682312e9388d4 I0702 02:17:12.759325 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=0c7afd87ad88dcb3cba76d60304a945e9a68c6165d4868eda2cdca8a4acc078f I0702 02:17:12.759381 1 recorder.go:70] Recording events/openshift-dns with fingerprint=8d6a97059658c1c908ce1cdf3d5c334a74ccfeafa48ff799b4f329029a0b1a01 I0702 02:17:12.759496 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=38c1b03b8304cdb08ad7997a7744cf63a9bf521a6d5bbc9b63a5c3c08f513501 I0702 02:17:12.759519 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=b49149cb92ed3056a974484a6015004245edcfe7142b8a75e883b549d419125d I0702 02:17:12.759587 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=5e55a8e35e5d984da137b01ea0493b4195a26db75c0938565f67d29bef568bbc I0702 02:17:12.759622 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=fd6e7c7d4db3ce1700f0024e31d0e6e9792567554f88634a505dc09347c6bd65 I0702 02:17:12.759774 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl with fingerprint=f35548f5b3308c8cc347ded739b8ccf825abe9d824cf4ad3aa736b3829bbdd78 E0702 02:17:12.759795 1 gather.go:165] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json" was already recorded and had the fingerprint "f35548f5b3308c8cc347ded739b8ccf825abe9d824cf4ad3aa736b3829bbdd78", overwriting with the record having fingerprint "f35548f5b3308c8cc347ded739b8ccf825abe9d824cf4ad3aa736b3829bbdd78" W0702 02:17:12.759807 1 gather.go:159] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json" because of the warning: warning: the record with the same fingerprint "f35548f5b3308c8cc347ded739b8ccf825abe9d824cf4ad3aa736b3829bbdd78" was already recorded at path "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json" I0702 02:17:12.759819 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-7d4ddc79d6-n6vxl/console-operator_previous.log with fingerprint=2dcacd44ee6506021f4921a86dae3888e104310b0e6dd6de0d7234eea526821f I0702 02:17:12.760052 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-7d4ddc79d6-n6vxl/console-operator_current.log with fingerprint=6fd3f0209ada0b670a4456a38e09c0ffa801e9d958dfdceec6396321fbd988ec I0702 02:17:12.760065 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-76d7c69695-5h574/console_current.log with fingerprint=15d4c6140a84659dc09d87fb5370b126e3ff406feb284d33b8299c28b78da786 I0702 02:17:12.760072 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-966565477-c8k29/console_current.log with fingerprint=a8c2b4278196560e547e2ea69fc4ed2dc2d7f30e753182ae07a2347955c17605 I0702 02:17:12.760080 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-966565477-cdzh2/console_current.log with fingerprint=38b0860535849b361b09e6024f72e5e967ebc9ad68a92e6599cb0f9f7469f8f2 I0702 02:17:12.760088 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-d9n2t/dns_current.log with fingerprint=cfb8fb9d3f021141c9964083ee574f62478bf205e9996f0a6bdda27ae1767887 I0702 02:17:12.760097 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-d9n2t/kube-rbac-proxy_current.log with fingerprint=f346455d00c0009e8ed1df36202a6e6242af765e251946fd09a69862d818be53 I0702 02:17:12.760208 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-n9jnm with fingerprint=b9d576c969fddc61eedebd8e695b2b7ace8a8b235fb717ade3cb704d1441e2fa I0702 02:17:12.760285 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-sw4fz with fingerprint=9218335fa398c4bec9af19b73208e6834381ce6c7411170259b6522305183019 I0702 02:17:12.760293 1 recorder.go:70] Recording config/pod/openshift-dns/logs/node-resolver-dr9ft/dns-node-resolver_current.log with fingerprint=dd1824ff14b82597ea1837e422d3a2de95e5f7b2d6930c8be56864c522c54422 I0702 02:17:12.760399 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-64d864fc7f-b9lks with fingerprint=c91b3b76a100e9108e3742be16fc6b58f7c0a434560a4c50cc56208205d670ac I0702 02:17:12.760439 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-6d74c9594-6ff7b/registry_current.log with fingerprint=49b791b80d9cfc2ba8c4812f6a3ded61cb69cf406069c8757c91cc4e767f34ad I0702 02:17:12.760475 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-6d74c9594-9449v/registry_current.log with fingerprint=6eb921de307a17c71c1d7d5275120be65dead72f7a721d9541a5ec890e0ed88b I0702 02:17:12.760480 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/node-ca-gj2q8/node-ca_current.log with fingerprint=4f7aff541b3b0e7b37625c9d3d0f208c4b3ef936c6770d9382d69eb001e2314b I0702 02:17:12.760495 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-655f54c855-522sw/router_current.log with fingerprint=a9dc5fcae1399b523c14206e79657907bed4ac951f3ec239d0db903ff7399fc3 I0702 02:17:12.760513 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-848bc4b65b-6csqh/router_current.log with fingerprint=0a7553a5fe96ecd187d3289a0622f41aff1c5a88421fa773bd598df1fde010e0 I0702 02:17:12.760539 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-848bc4b65b-hvvmq/router_current.log with fingerprint=60d12e07a48de07fe2e03171576a128fcc7c025ac7b868282f298d1fb7b60133 I0702 02:17:12.760606 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-927fr with fingerprint=61af3d3a5c11022161513fb32d1f9b5032a4124f5f3274b6a08c8a40d3ad15ef I0702 02:17:12.760690 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-hjjfs with fingerprint=a9a8dba3af9c2427c9540c6fead176e540351ab0a9baa71813bc5097fc4ac269 I0702 02:17:12.760702 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-vdnpm/serve-healthcheck-canary_current.log with fingerprint=f0bc4e512c74df49f3f4a65e99a92384da256f3d936963085da27bbf2a9b1fa9 I0702 02:17:12.760710 1 gather.go:181] gatherer "clusterconfig" function "operators_pods_and_events" took 7.041541147s to process 29 records I0702 02:17:18.734806 1 tasks_processing.go:74] worker 12 stopped. I0702 02:17:18.734875 1 recorder.go:70] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0702 02:17:18.734892 1 gather.go:181] gatherer "clusterconfig" function "install_plans" took 13.028684638s to process 1 records I0702 02:17:19.490269 1 tasks_processing.go:74] worker 28 stopped. I0702 02:17:19.490585 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=ba16ea6b4503917d33437d4d1999039313b5aa3c669ab94c714d5681918865aa I0702 02:17:19.490602 1 gather.go:181] gatherer "clusterconfig" function "service_accounts" took 13.807600026s to process 1 records E0702 02:17:19.490665 1 periodic.go:252] clusterconfig failed after 13.809s with: function "metrics" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "active_alerts" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json" I0702 02:17:19.490689 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "metrics" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "active_alerts" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-n6vxl.json" I0702 02:17:19.490699 1 periodic.go:214] Running workloads gatherer I0702 02:17:19.490712 1 tasks_processing.go:45] number of workers: 2 I0702 02:17:19.490723 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 02:17:19.490727 1 tasks_processing.go:71] worker 1 working on workload_info task. I0702 02:17:19.490739 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 02:17:19.490814 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0702 02:17:19.513739 1 gather_workloads_info.go:281] Loaded pods in 0s, will wait 24s for image data I0702 02:17:19.524394 1 tasks_processing.go:74] worker 0 stopped. I0702 02:17:19.524409 1 gather.go:181] gatherer "workloads" function "helmchart_info" took 33.563612ms to process 0 records I0702 02:17:19.524608 1 gather_workloads_info.go:390] No image sha256:0c1abef76d6d2832ddcff31e950ffafeb4e0f6fad80a5f5dca94591de0fc9581 (12ms) I0702 02:17:19.535875 1 gather_workloads_info.go:390] No image sha256:5bcfb1b739fa4e6ff9e21e4843f8520f29608135341f7d8dcc31084b9ba25f1d (11ms) I0702 02:17:19.547668 1 gather_workloads_info.go:390] No image sha256:b8781637560d49de9b8e8cdd59ec75f03d98921ea66950381849398128cc9cb9 (12ms) I0702 02:17:19.559249 1 gather_workloads_info.go:390] No image sha256:48dff3fcb0da58eb9b2b500f89af8b7d0790ffa60da0b0e301feca971dbafcbb (12ms) I0702 02:17:19.571335 1 gather_workloads_info.go:390] No image sha256:02283a64c1936a5dd09d5c6bc0ecbcf9196fe4df6a4055c8be22cebd850aa7a3 (12ms) I0702 02:17:19.583089 1 gather_workloads_info.go:390] No image sha256:4dcdadc14b7bdb2c00954d92e957d63aafbbe5f4047fe028e575d65725923cb0 (12ms) I0702 02:17:19.594678 1 gather_workloads_info.go:390] No image sha256:09b3e9e6a4f382fb4bbdb67bc945bcf19d66f604b7c8ffd66d4d4a2c94224248 (12ms) I0702 02:17:19.606450 1 gather_workloads_info.go:390] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (12ms) I0702 02:17:19.618198 1 gather_workloads_info.go:390] No image sha256:4b9b7949d209e257a76053de4ac74193f747c0d42e25dcd270d655849a4e51aa (12ms) I0702 02:17:19.629598 1 gather_workloads_info.go:390] No image sha256:41029e57ece558b2e0b12174ce924647fd9d5c5d83aadf4bba03d6916d3f9f54 (11ms) I0702 02:17:19.641097 1 gather_workloads_info.go:390] No image sha256:788d16bfff4bd6bb3ce97acc643b56fae6e619674ab279535480e519b4d5b769 (11ms) I0702 02:17:19.725307 1 gather_workloads_info.go:390] No image sha256:30ff5e5b72f274598704ca69f1832eba1040cc25da47386004fd59500b6e4c55 (84ms) I0702 02:17:19.825742 1 gather_workloads_info.go:390] No image sha256:2897724cec17bc13acc133181f64ccb8d54ce5dc9582f9cecf704c2540e15600 (100ms) I0702 02:17:19.925238 1 gather_workloads_info.go:390] No image sha256:1c7c24111f1709d7648156873ced09462638aa4b3cf7e7aee13297e51bcdfb3e (99ms) I0702 02:17:20.027658 1 gather_workloads_info.go:390] No image sha256:13aabbaf260d634ff0e9ac1af88af075b5ba671fd51313e33e74d7d2f7e54128 (102ms) I0702 02:17:20.125527 1 gather_workloads_info.go:390] No image sha256:0158b297aec0a66376c04d3c7cf4828db5e5d9f424e331ff7c7b796828036875 (98ms) I0702 02:17:20.225867 1 gather_workloads_info.go:390] No image sha256:221622a101db9797df0622b4534e3e42c4498df458f355a8a02fad8d65f8e3f2 (100ms) I0702 02:17:20.325430 1 gather_workloads_info.go:390] No image sha256:42b4c6bdf3081ec264bc5edbc4eb779151de9e4742d6fe59a316645891df649c (100ms) I0702 02:17:20.425151 1 gather_workloads_info.go:390] No image sha256:06821dfe231330e0eb5be915b9468cd13826a7ee59d1a732b89cb140548eade2 (100ms) I0702 02:17:20.525532 1 gather_workloads_info.go:390] No image sha256:351ca92471647f383f3cd7f0b70f33f5fa11c39ab569c4948dff7d471a6c8be3 (100ms) I0702 02:17:20.624905 1 gather_workloads_info.go:390] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (99ms) I0702 02:17:20.725167 1 gather_workloads_info.go:390] No image sha256:3fea7db586d3236c1876a8b523af0d8d65c84d6f6cf5d0f4e41cafcffe6db5b5 (100ms) I0702 02:17:20.825981 1 gather_workloads_info.go:390] No image sha256:38787de16ee8c686305f93b501d891b50cde24136009863869d230c0d4eeee5c (101ms) I0702 02:17:20.925366 1 gather_workloads_info.go:390] No image sha256:31a22f6ee061d0f6558664e580ec0f80f1927d4ec29626cf3e9420837f5d80e5 (99ms) I0702 02:17:21.025699 1 gather_workloads_info.go:390] No image sha256:bc829520eeec090b60d17abbad1a22b34ea99b426193395808a778d73a5fb57a (100ms) I0702 02:17:21.125124 1 gather_workloads_info.go:390] No image sha256:1eee856bde9bad71c4444c494225a0cf02d1f4a5293acf0bd6e7e1814efa0064 (99ms) I0702 02:17:21.224727 1 gather_workloads_info.go:390] No image sha256:0c6b7f828184b16a5e6630111e0715e11d34dae9b0bf64619466543e6168e05c (100ms) I0702 02:17:21.325623 1 gather_workloads_info.go:390] No image sha256:194ee1155b87e110bc6c4a6e55667e41f3bdc8292e26a9561f9082fdfe467eec (101ms) I0702 02:17:21.425097 1 gather_workloads_info.go:390] No image sha256:a7fc4d05c4a3c82c9f355ca985d6fd26033c0d6a98716f25fcd2f54bae1d2fa0 (99ms) I0702 02:17:21.526017 1 gather_workloads_info.go:390] No image sha256:8562a483f4568254250f19fc68d79f88c94a48508aefe59a70c1021a3aad000a (101ms) I0702 02:17:21.625942 1 gather_workloads_info.go:390] No image sha256:ed0c9631d6e424912cd8ce78ca7a2c4f76b3fc15e9e34fe00999170ccf19a611 (100ms) I0702 02:17:21.725188 1 gather_workloads_info.go:390] No image sha256:c7aa910ca06f8fd89cdf2519136e0f3533bdacfc6bd58a3f6683f9f08756c7b1 (99ms) I0702 02:17:21.826785 1 gather_workloads_info.go:390] No image sha256:0242cefbf0bb876101c4d67ab4e07401cb0ba6c02be2b73a4f17ba33b2960005 (102ms) I0702 02:17:21.924729 1 gather_workloads_info.go:390] No image sha256:8e158c83ec12b3e0237fab9a221630bd691e79c079a111f346354ff43b8199df (98ms) I0702 02:17:22.025486 1 gather_workloads_info.go:390] No image sha256:3969660e807e8ce5656a74e5816573656241946ef1d766a9d2e21d434356313c (101ms) I0702 02:17:22.125224 1 gather_workloads_info.go:390] No image sha256:3a2c82b772ea6072e1eca0efc5303f002189e80b0e985fb3e3b8ea9a12fe8060 (100ms) I0702 02:17:22.231571 1 gather_workloads_info.go:390] No image sha256:68abf16a3b6c33fdd22a823c7d00f96aaaa8015d370116962a5835984908645a (106ms) I0702 02:17:22.329743 1 gather_workloads_info.go:390] No image sha256:586127c49e25052113263d8aeaf6a1623aef22987364f119a17b2848982ff716 (98ms) I0702 02:17:22.426769 1 gather_workloads_info.go:390] No image sha256:50076d29400ecf9b704d8175509fc8cb7c28bea91d46a28dc37cbda1668c69f9 (97ms) I0702 02:17:22.526504 1 gather_workloads_info.go:390] No image sha256:894452e6587de55d04e37163d3da53e11fced8fb08bd0acf61e6d49f7964284e (100ms) I0702 02:17:22.625121 1 gather_workloads_info.go:390] No image sha256:751bd7eb62f632c0eef0ca91b6c57c64d4f1279d4dbe1a9b875ac4020a7b693f (99ms) I0702 02:17:22.726315 1 gather_workloads_info.go:390] No image sha256:c96b4bddf3dd7daaa797cbda4dd92e981de2f5fc3404c01381f31dc0414d5c8d (101ms) I0702 02:17:22.827348 1 gather_workloads_info.go:390] No image sha256:24519dcc0e63cdc60a639dd474d06e7612e9b76966bc60322cd9026a0f1b878c (101ms) I0702 02:17:22.926255 1 gather_workloads_info.go:390] No image sha256:6a9b4fd887fa7f1729302fb0330c3c9a483197bf78becdcfda4235ec823a2c39 (99ms) I0702 02:17:22.926286 1 tasks_processing.go:74] worker 1 stopped. I0702 02:17:22.926601 1 recorder.go:70] Recording config/workload_info with fingerprint=8b25132505242e29b1132b076c937d61eebbf08e1abd24e7ea8698501314e5c1 I0702 02:17:22.926617 1 gather.go:181] gatherer "workloads" function "workload_info" took 3.43555276s to process 1 records I0702 02:17:22.926630 1 periodic.go:261] Periodic gather workloads completed in 3.435s I0702 02:17:22.926639 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0702 02:17:22.926644 1 periodic.go:214] Running conditional gatherer I0702 02:17:22.936324 1 requests.go:282] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.18.9/gathering_rules I0702 02:17:23.056823 1 conditional_gatherer.go:336] updating alerts cache for conditional gatherer E0702 02:17:23.059619 1 conditional_gatherer.go:322] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.178.138:9091: connect: connection refused I0702 02:17:23.059698 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0702 02:17:23.070050 1 conditional_gatherer.go:390] cluster version is '4.18.9' E0702 02:17:23.070066 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070074 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070079 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070083 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070088 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070094 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070098 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070102 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070106 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:17:23.070112 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0702 02:17:23.070131 1 tasks_processing.go:45] number of workers: 3 I0702 02:17:23.070155 1 tasks_processing.go:69] worker 2 listening for tasks. I0702 02:17:23.070165 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0702 02:17:23.070163 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 02:17:23.070173 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 02:17:23.070178 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0702 02:17:23.070184 1 tasks_processing.go:74] worker 1 stopped. I0702 02:17:23.070193 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0702 02:17:23.070270 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0702 02:17:23.070287 1 gather.go:181] gatherer "conditional" function "conditional_gatherer_rules" took 1.272µs to process 1 records I0702 02:17:23.070332 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0702 02:17:23.070343 1 gather.go:181] gatherer "conditional" function "remote_configuration" took 1.489µs to process 1 records I0702 02:17:23.070352 1 tasks_processing.go:74] worker 0 stopped. I0702 02:17:23.070402 1 tasks_processing.go:74] worker 2 stopped. I0702 02:17:23.070414 1 gather.go:181] gatherer "conditional" function "rapid_container_logs" took 190.393µs to process 0 records I0702 02:17:23.070430 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=AsExpected message= I0702 02:17:23.070450 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f W0702 02:17:23.102593 1 gather.go:213] can't read cgroups memory usage data: open /sys/fs/cgroup/memory/memory.usage_in_bytes: no such file or directory I0702 02:17:23.102776 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=2d8ccfc3952cdbe8276bb2548d129f415f235743dcfa842fa15348561566c9f2 I0702 02:17:23.102924 1 diskrecorder.go:70] Writing 132 records to /var/lib/insights-operator/insights-2026-07-02-021723.tar.gz I0702 02:17:23.114343 1 diskrecorder.go:51] Wrote 132 records to disk in 11ms I0702 02:17:23.114378 1 periodic.go:283] Gathering cluster info every 2h0m0s I0702 02:17:23.114394 1 periodic.go:284] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0702 02:18:35.681736 1 diskrecorder.go:170] Found files to send: insights-2026-07-02-021723.tar.gz I0702 02:18:35.681763 1 insightsuploader.go:150] Checking archives to upload periodically every 15m46.613691221s I0702 02:18:35.681771 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0702 02:18:35.699065 1 requests.go:47] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0702 02:18:35.907176 1 requests.go:88] Successfully reported id=2026-07-02T02:18:35Z x-rh-insights-request-id=bece9429932746a9a5a10f48bd7a6513, wrote=74948 I0702 02:18:35.907230 1 insightsuploader.go:187] Uploaded report successfully in 225.452044ms I0702 02:18:35.907254 1 controller.go:119] Initializing last reported time to 2026-07-02T02:18:35Z I0702 02:18:35.907287 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0702 02:18:35.907297 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0702 02:18:35.907304 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0702 02:18:35.914734 1 controller.go:482] The operator is healthy I0702 02:19:05.705279 1 controller.go:482] The operator is healthy I0702 02:19:40.650437 1 insightsreport.go:137] Pulling report from smart-proxy I0702 02:19:40.650463 1 insightsreport.go:149] Retrieving report I0702 02:19:40.661928 1 requests.go:111] Retrieving report for cluster: 8af3aa7e-1984-4c38-b1ca-712be8222f0b I0702 02:19:40.661941 1 requests.go:112] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/8af3aa7e-1984-4c38-b1ca-712be8222f0b/reports I0702 02:19:40.664812 1 requests.go:122] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/8af3aa7e-1984-4c38-b1ca-712be8222f0b/reports I0702 02:19:40.871662 1 insightsreport.go:184] Report retrieved I0702 02:19:40.921169 1 insightsreport.go:239] Report retrieved correctly I0702 02:21:05.710912 1 controller.go:482] The operator is healthy I0702 02:22:05.679270 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:22:05.772610 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:22:05.772639 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:22:05.803375 1 secretconfigobserver.go:119] support secret does not exist I0702 02:23:05.707470 1 controller.go:482] The operator is healthy I0702 02:25:05.706988 1 controller.go:482] The operator is healthy I0702 02:27:05.679903 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:27:05.685988 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:27:05.704154 1 controller.go:482] The operator is healthy I0702 02:27:05.804395 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:27:05.810983 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:27:05.811009 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:27:05.816905 1 secretconfigobserver.go:119] support secret does not exist I0702 02:29:05.706774 1 controller.go:482] The operator is healthy I0702 02:31:05.711477 1 controller.go:482] The operator is healthy I0702 02:32:05.817785 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:32:05.827349 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:32:05.827374 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:32:05.834084 1 secretconfigobserver.go:119] support secret does not exist I0702 02:33:05.707381 1 controller.go:482] The operator is healthy I0702 02:34:27.521966 1 insightsuploader.go:146] Nothing to report since 2026-07-02T02:18:35Z I0702 02:35:05.704250 1 controller.go:482] The operator is healthy I0702 02:37:05.676084 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:37:05.704094 1 controller.go:482] The operator is healthy I0702 02:37:05.834162 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:37:05.840575 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:37:05.840600 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:37:05.846428 1 secretconfigobserver.go:119] support secret does not exist I0702 02:39:05.715124 1 controller.go:482] The operator is healthy I0702 02:41:05.704314 1 controller.go:482] The operator is healthy I0702 02:42:05.847459 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:42:05.858206 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:42:05.858228 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:42:05.864153 1 secretconfigobserver.go:119] support secret does not exist I0702 02:43:05.708495 1 controller.go:482] The operator is healthy I0702 02:45:05.707371 1 controller.go:482] The operator is healthy I0702 02:47:05.676581 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:47:05.682396 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:47:05.703704 1 controller.go:482] The operator is healthy I0702 02:47:05.864991 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:47:05.871796 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:47:05.871825 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:47:05.877612 1 secretconfigobserver.go:119] support secret does not exist I0702 02:49:05.707977 1 controller.go:482] The operator is healthy I0702 02:50:08.586002 1 insightsuploader.go:146] Nothing to report since 2026-07-02T02:18:35Z I0702 02:51:05.708666 1 controller.go:482] The operator is healthy I0702 02:52:05.878118 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:52:05.885005 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:52:05.885028 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:52:05.890907 1 secretconfigobserver.go:119] support secret does not exist I0702 02:53:05.704428 1 controller.go:482] The operator is healthy I0702 02:55:05.707550 1 controller.go:482] The operator is healthy I0702 02:57:05.678727 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:57:05.705158 1 controller.go:482] The operator is healthy I0702 02:57:05.891349 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:57:05.899126 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:57:05.899157 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:57:05.905449 1 secretconfigobserver.go:119] support secret does not exist