I0702 02:13:20.139472 1 cmd.go:250] Using service-serving-cert provided certificates I0702 02:13:20.139876 1 observer_polling.go:159] Starting file observer W0702 02:13:20.161030 1 builder.go:266] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-6fd96f47fd" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0702 02:13:20.905488 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0702 02:13:20.905732 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0702 02:13:20.906200 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0702 02:13:20.906316 1 secure_serving.go:57] Forcing use of http/1.1 only W0702 02:13:20.906341 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0702 02:13:20.906346 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0702 02:13:20.906350 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0702 02:13:20.906352 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0702 02:13:20.906355 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0702 02:13:20.906357 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0702 02:13:20.909989 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes ChunkSizeMiB CloudDualStackNodeIPs DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy PersistentIPsForVirtualization PrivateHostedZoneAWS SetEIPForNLBIngressController VSphereControlPlaneMachineSet VSphereDriverConfiguration VSphereMultiVCenters VSphereStaticIPs ValidatingAdmissionPolicy AWSClusterHostedDNS AdditionalRoutingCapabilities AutomatedEtcdBackup BootcNodeManagement CSIDriverSharedResource ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ConsolePluginContentSecurityPolicy DNSNameResolver DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example GCPClusterHostedDNS GatewayAPI ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MetricsCollectionProfiles MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NodeSwap NutanixMultiSubnets OVNObservability OnClusterBuild OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SignatureStores SigstoreImageVerification TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0702 02:13:20.909998 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "ChunkSizeMiB", "CloudDualStackNodeIPs", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "SetEIPForNLBIngressController", "VSphereControlPlaneMachineSet", "VSphereDriverConfiguration", "VSphereMultiVCenters", "VSphereStaticIPs", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AdditionalRoutingCapabilities", "AutomatedEtcdBackup", "BootcNodeManagement", "CSIDriverSharedResource", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ConsolePluginContentSecurityPolicy", "DNSNameResolver", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "GCPClusterHostedDNS", "GatewayAPI", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MetricsCollectionProfiles", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OnClusterBuild", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SignatureStores", "SigstoreImageVerification", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0702 02:13:20.911980 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController I0702 02:13:20.912008 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0702 02:13:20.912020 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0702 02:13:20.912028 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0702 02:13:20.912067 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0702 02:13:20.912079 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0702 02:13:20.912876 1 secure_serving.go:213] Serving securely on [::]:8443 I0702 02:13:20.913020 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0702 02:13:20.913184 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" E0702 02:13:20.914086 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18be573817896961 openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"SetEIPForNLBIngressController\", \"VSphereControlPlaneMachineSet\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"VSphereStaticIPs\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AdditionalRoutingCapabilities\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"CSIDriverSharedResource\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ConsolePluginContentSecurityPolicy\", \"DNSNameResolver\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"GCPClusterHostedDNS\", \"GatewayAPI\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MetricsCollectionProfiles\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OnClusterBuild\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SignatureStores\", \"SigstoreImageVerification\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-07-02 02:13:20.909961569 +0000 UTC m=+0.811754751,LastTimestamp:2026-07-02 02:13:20.909961569 +0000 UTC m=+0.811754751,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W0702 02:13:20.914951 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0702 02:13:20.914979 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0702 02:13:20.915079 1 base_controller.go:67] Waiting for caches to sync for ConfigController I0702 02:13:20.919452 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:13:20.919470 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0702 02:13:20.925323 1 secretconfigobserver.go:119] support secret does not exist I0702 02:13:20.929871 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:13:20.931304 1 recorder.go:156] Pruning old reports every 5h57m25s, max age is 288h0m0s I0702 02:13:20.935730 1 secretconfigobserver.go:119] support secret does not exist I0702 02:13:20.936821 1 periodic.go:214] Running clusterconfig gatherer I0702 02:13:20.936827 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0702 02:13:20.936841 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0702 02:13:20.936857 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0702 02:13:20.937043 1 insightsreport.go:296] Starting report retriever I0702 02:13:20.937094 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0702 02:13:20.937122 1 tasks_processing.go:45] number of workers: 32 I0702 02:13:20.937245 1 tasks_processing.go:69] worker 31 listening for tasks. I0702 02:13:20.937310 1 tasks_processing.go:71] worker 31 working on nodenetworkconfigurationpolicies task. I0702 02:13:20.937618 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 02:13:20.937664 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 02:13:20.937703 1 tasks_processing.go:69] worker 2 listening for tasks. I0702 02:13:20.937731 1 tasks_processing.go:69] worker 3 listening for tasks. I0702 02:13:20.937770 1 tasks_processing.go:69] worker 4 listening for tasks. I0702 02:13:20.937797 1 tasks_processing.go:69] worker 5 listening for tasks. I0702 02:13:20.937823 1 tasks_processing.go:69] worker 6 listening for tasks. I0702 02:13:20.937850 1 tasks_processing.go:69] worker 7 listening for tasks. I0702 02:13:20.937878 1 tasks_processing.go:69] worker 8 listening for tasks. I0702 02:13:20.937905 1 tasks_processing.go:69] worker 9 listening for tasks. I0702 02:13:20.937932 1 tasks_processing.go:69] worker 10 listening for tasks. I0702 02:13:20.937959 1 tasks_processing.go:69] worker 11 listening for tasks. I0702 02:13:20.937991 1 tasks_processing.go:69] worker 12 listening for tasks. I0702 02:13:20.938018 1 tasks_processing.go:69] worker 13 listening for tasks. I0702 02:13:20.938044 1 tasks_processing.go:69] worker 14 listening for tasks. I0702 02:13:20.938070 1 tasks_processing.go:69] worker 15 listening for tasks. I0702 02:13:20.938097 1 tasks_processing.go:69] worker 16 listening for tasks. I0702 02:13:20.938123 1 tasks_processing.go:69] worker 17 listening for tasks. I0702 02:13:20.938151 1 tasks_processing.go:69] worker 18 listening for tasks. I0702 02:13:20.938178 1 tasks_processing.go:69] worker 19 listening for tasks. I0702 02:13:20.938208 1 tasks_processing.go:69] worker 20 listening for tasks. I0702 02:13:20.938273 1 tasks_processing.go:69] worker 21 listening for tasks. I0702 02:13:20.938300 1 tasks_processing.go:69] worker 22 listening for tasks. I0702 02:13:20.938331 1 tasks_processing.go:69] worker 25 listening for tasks. I0702 02:13:20.938377 1 tasks_processing.go:71] worker 20 working on image task. I0702 02:13:20.938405 1 tasks_processing.go:71] worker 25 working on cluster_apiserver task. I0702 02:13:20.938412 1 tasks_processing.go:71] worker 9 working on crds task. I0702 02:13:20.938415 1 tasks_processing.go:71] worker 17 working on sap_config task. I0702 02:13:20.938418 1 tasks_processing.go:71] worker 13 working on support_secret task. I0702 02:13:20.938334 1 tasks_processing.go:69] worker 27 listening for tasks. I0702 02:13:20.938472 1 tasks_processing.go:71] worker 27 working on nodenetworkstates task. I0702 02:13:20.938333 1 tasks_processing.go:69] worker 26 listening for tasks. I0702 02:13:20.938337 1 tasks_processing.go:69] worker 24 listening for tasks. I0702 02:13:20.938350 1 tasks_processing.go:69] worker 28 listening for tasks. I0702 02:13:20.938350 1 tasks_processing.go:69] worker 29 listening for tasks. I0702 02:13:20.938353 1 tasks_processing.go:69] worker 30 listening for tasks. I0702 02:13:20.938354 1 tasks_processing.go:69] worker 23 listening for tasks. I0702 02:13:20.938361 1 tasks_processing.go:71] worker 0 working on ceph_cluster task. I0702 02:13:20.938528 1 tasks_processing.go:71] worker 23 working on tsdb_status task. I0702 02:13:20.938564 1 tasks_processing.go:71] worker 28 working on certificate_signing_requests task. I0702 02:13:20.938623 1 tasks_processing.go:71] worker 26 working on monitoring_persistent_volumes task. I0702 02:13:20.938642 1 tasks_processing.go:71] worker 29 working on openstack_version task. I0702 02:13:20.938364 1 tasks_processing.go:71] worker 6 working on image_pruners task. I0702 02:13:20.938413 1 tasks_processing.go:71] worker 12 working on container_runtime_configs task. I0702 02:13:20.938834 1 tasks_processing.go:71] worker 24 working on image_registries task. I0702 02:13:20.938365 1 tasks_processing.go:71] worker 3 working on storage_classes task. I0702 02:13:20.939122 1 tasks_processing.go:71] worker 30 working on networks task. I0702 02:13:20.938365 1 tasks_processing.go:71] worker 4 working on overlapping_namespace_uids task. I0702 02:13:20.938371 1 tasks_processing.go:71] worker 22 working on cost_management_metrics_configs task. I0702 02:13:20.938370 1 tasks_processing.go:71] worker 5 working on machine_configs task. I0702 02:13:20.938375 1 tasks_processing.go:71] worker 14 working on install_plans task. I0702 02:13:20.938372 1 tasks_processing.go:71] worker 2 working on mutating_webhook_configurations task. I0702 02:13:20.938376 1 tasks_processing.go:71] worker 18 working on openshift_logging task. I0702 02:13:20.938377 1 tasks_processing.go:71] worker 19 working on machine_sets task. I0702 02:13:20.938401 1 tasks_processing.go:71] worker 7 working on openstack_dataplanedeployments task. I0702 02:13:20.938401 1 tasks_processing.go:71] worker 21 working on validating_webhook_configurations task. I0702 02:13:20.938404 1 tasks_processing.go:71] worker 15 working on pdbs task. I0702 02:13:20.938376 1 tasks_processing.go:71] worker 10 working on proxies task. I0702 02:13:20.938407 1 tasks_processing.go:71] worker 8 working on container_images task. I0702 02:13:20.938409 1 tasks_processing.go:71] worker 16 working on machine_autoscalers task. I0702 02:13:20.938420 1 tasks_processing.go:71] worker 11 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0702 02:13:20.938361 1 tasks_processing.go:71] worker 1 working on node_logs task. I0702 02:13:20.941936 1 tasks_processing.go:71] worker 31 working on machine_healthchecks task. I0702 02:13:20.941945 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 4.587608ms to process 0 records E0702 02:13:20.943569 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:48877->172.30.0.10:53: read: connection refused I0702 02:13:20.943591 1 tasks_processing.go:71] worker 23 working on oauths task. E0702 02:13:20.943602 1 gather.go:144] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:48877->172.30.0.10:53: read: connection refused I0702 02:13:20.943622 1 gather.go:181] gatherer "clusterconfig" function "tsdb_status" took 5.052633ms to process 0 records I0702 02:13:20.944762 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0702 02:13:20.944778 1 controller.go:317] The initial operator extension status is healthy I0702 02:13:20.944785 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0702 02:13:20.944790 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0702 02:13:20.944794 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0702 02:13:20.944808 1 controller.go:457] The operator is still being initialized I0702 02:13:20.944815 1 controller.go:482] The operator is healthy I0702 02:13:20.944894 1 sca.go:98] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/certificates. Next check is in 8h0m0s I0702 02:13:20.944946 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0702 02:13:20.945058 1 operator.go:288] started I0702 02:13:20.945085 1 base_controller.go:67] Waiting for caches to sync for LoggingSyncer I0702 02:13:20.945327 1 tasks_processing.go:71] worker 31 working on version task. E0702 02:13:20.945347 1 gather.go:144] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0702 02:13:20.945357 1 gather.go:181] gatherer "clusterconfig" function "machine_healthchecks" took 3.374491ms to process 0 records I0702 02:13:20.945420 1 tasks_processing.go:71] worker 17 working on sap_pods task. I0702 02:13:20.945429 1 gather.go:181] gatherer "clusterconfig" function "sap_config" took 6.986784ms to process 0 records I0702 02:13:20.946115 1 tasks_processing.go:71] worker 12 working on dvo_metrics task. I0702 02:13:20.946127 1 gather.go:181] gatherer "clusterconfig" function "container_runtime_configs" took 7.27438ms to process 0 records I0702 02:13:20.947749 1 tasks_processing.go:71] worker 27 working on operators task. I0702 02:13:20.947757 1 gather.go:181] gatherer "clusterconfig" function "nodenetworkstates" took 9.263018ms to process 0 records I0702 02:13:20.948032 1 tasks_processing.go:71] worker 0 working on pod_network_connectivity_checks task. I0702 02:13:20.948043 1 gather.go:181] gatherer "clusterconfig" function "ceph_cluster" took 9.504997ms to process 0 records I0702 02:13:20.948946 1 tasks_processing.go:71] worker 29 working on storage_cluster task. I0702 02:13:20.948957 1 gather.go:181] gatherer "clusterconfig" function "openstack_version" took 10.290581ms to process 0 records I0702 02:13:20.949418 1 tasks_processing.go:71] worker 16 working on sap_datahubs task. I0702 02:13:20.949429 1 gather.go:181] gatherer "clusterconfig" function "machine_autoscalers" took 9.347359ms to process 0 records I0702 02:13:20.949440 1 gather.go:181] gatherer "clusterconfig" function "cost_management_metrics_configs" took 10.119015ms to process 0 records I0702 02:13:20.949447 1 tasks_processing.go:71] worker 22 working on machine_config_pools task. I0702 02:13:20.949624 1 tasks_processing.go:71] worker 18 working on silenced_alerts task. I0702 02:13:20.949635 1 gather.go:181] gatherer "clusterconfig" function "openshift_logging" took 10.052154ms to process 0 records I0702 02:13:20.949882 1 tasks_processing.go:71] worker 7 working on openstack_dataplanenodesets task. I0702 02:13:20.949926 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 10.175382ms to process 0 records I0702 02:13:20.951361 1 tasks_processing.go:71] worker 19 working on machines task. I0702 02:13:20.951372 1 gather.go:181] gatherer "clusterconfig" function "machine_sets" took 11.754577ms to process 0 records I0702 02:13:20.951664 1 tasks_processing.go:71] worker 23 working on clusterroles task. I0702 02:13:20.952605 1 recorder.go:70] Recording config/oauth with fingerprint=f79505dedd6ff9cc17f41d8da85597c0a923e9effafd847368ae05fc84cafa85 I0702 02:13:20.952626 1 gather.go:181] gatherer "clusterconfig" function "oauths" took 8.059948ms to process 1 records E0702 02:13:20.953581 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:58075->172.30.0.10:53: read: connection refused I0702 02:13:20.953599 1 tasks_processing.go:71] worker 18 working on openstack_controlplanes task. E0702 02:13:20.953634 1 gather.go:144] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:58075->172.30.0.10:53: read: connection refused I0702 02:13:20.953655 1 gather.go:181] gatherer "clusterconfig" function "silenced_alerts" took 3.964481ms to process 0 records I0702 02:13:20.953975 1 tasks_processing.go:71] worker 0 working on config_maps task. E0702 02:13:20.953987 1 gather.go:144] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0702 02:13:20.953994 1 gather.go:181] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 5.927637ms to process 0 records E0702 02:13:20.954009 1 gather.go:144] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0702 02:13:20.954059 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0702 02:13:20.954075 1 gather.go:181] gatherer "clusterconfig" function "machine_configs" took 14.660344ms to process 1 records I0702 02:13:20.954088 1 gather.go:181] gatherer "clusterconfig" function "storage_cluster" took 5.127009ms to process 0 records I0702 02:13:20.954125 1 tasks_processing.go:71] worker 29 working on operators_pods_and_events task. I0702 02:13:20.954149 1 tasks_processing.go:71] worker 5 working on feature_gates task. I0702 02:13:20.954249 1 tasks_processing.go:71] worker 10 working on authentication task. I0702 02:13:20.954352 1 recorder.go:70] Recording config/proxy with fingerprint=5b4b68da0c86b9cd11cd3e3d19341f1ef680fb0c28cf7806b6aaa2b705de704e I0702 02:13:20.954366 1 gather.go:181] gatherer "clusterconfig" function "proxies" took 14.313833ms to process 1 records I0702 02:13:20.954379 1 gather.go:181] gatherer "clusterconfig" function "sap_datahubs" took 4.836011ms to process 0 records I0702 02:13:20.954388 1 tasks_processing.go:71] worker 16 working on nodes task. I0702 02:13:20.954837 1 tasks_processing.go:71] worker 13 working on lokistack task. E0702 02:13:20.954849 1 gather.go:144] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0702 02:13:20.954860 1 gather.go:181] gatherer "clusterconfig" function "support_secret" took 16.404474ms to process 0 records I0702 02:13:20.958510 1 tasks_processing.go:71] worker 22 working on openshift_machine_api_events task. I0702 02:13:20.958519 1 gather.go:181] gatherer "clusterconfig" function "machine_config_pools" took 9.052682ms to process 0 records I0702 02:13:20.958718 1 tasks_processing.go:71] worker 30 working on ingress_certificates task. I0702 02:13:20.958912 1 recorder.go:70] Recording config/network with fingerprint=14c65d25cdb163e5bd603b7c82f807cc6e983e2e52504726f0b7109ee8742b2a I0702 02:13:20.958924 1 gather.go:181] gatherer "clusterconfig" function "networks" took 19.555746ms to process 1 records I0702 02:13:20.960774 1 tasks_processing.go:71] worker 24 working on infrastructures task. I0702 02:13:20.961561 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=b1af288c616dfdf4016d490437de8fb2ad1874e86df208675008652ddafbbb94 I0702 02:13:20.961583 1 gather.go:181] gatherer "clusterconfig" function "image_registries" took 21.925122ms to process 1 records E0702 02:13:20.961600 1 gather.go:144] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0702 02:13:20.961610 1 gather.go:181] gatherer "clusterconfig" function "machines" took 9.504844ms to process 0 records I0702 02:13:20.961618 1 tasks_processing.go:71] worker 19 working on metrics task. I0702 02:13:20.964924 1 tasks_processing.go:71] worker 18 working on active_alerts task. I0702 02:13:20.964941 1 gather.go:181] gatherer "clusterconfig" function "openstack_controlplanes" took 11.310724ms to process 0 records I0702 02:13:20.965290 1 tasks_processing.go:71] worker 17 working on aggregated_monitoring_cr_names task. I0702 02:13:20.965370 1 gather.go:181] gatherer "clusterconfig" function "sap_pods" took 19.856483ms to process 0 records I0702 02:13:20.965583 1 tasks_processing.go:71] worker 20 working on olm_operators task. I0702 02:13:20.965590 1 recorder.go:70] Recording config/image with fingerprint=b80ada6fa7faf79d3edb9b1a858c632430ac40364a5bad7007e6933e8bd61408 I0702 02:13:20.965618 1 gather.go:181] gatherer "clusterconfig" function "image" took 26.951067ms to process 1 records I0702 02:13:20.965709 1 tasks_processing.go:71] worker 6 working on service_accounts task. I0702 02:13:20.965714 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=c7b0dbd9d106833d01857cca7d8e42a4cd099c50bc959734bb621be184dc7ba7 I0702 02:13:20.965930 1 gather.go:181] gatherer "clusterconfig" function "image_pruners" took 26.637655ms to process 1 records I0702 02:13:20.966036 1 tasks_processing.go:71] worker 25 working on jaegers task. E0702 02:13:20.966561 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:34533->172.30.0.10:53: read: connection refused I0702 02:13:20.967621 1 recorder.go:70] Recording config/apiserver with fingerprint=58fa3ff7fc6fcd4fd362d12a9d24174acd53f4d2494f7be19b54606d80eb8f0c I0702 02:13:20.967642 1 gather.go:181] gatherer "clusterconfig" function "cluster_apiserver" took 27.069619ms to process 1 records I0702 02:13:20.967652 1 gather.go:181] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 15.606504ms to process 0 records I0702 02:13:20.967658 1 gather.go:181] gatherer "clusterconfig" function "lokistack" took 11.452061ms to process 0 records E0702 02:13:20.967667 1 gather.go:144] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:34533->172.30.0.10:53: read: connection refused I0702 02:13:20.967690 1 tasks_processing.go:71] worker 7 working on schedulers task. I0702 02:13:20.967696 1 gather.go:181] gatherer "clusterconfig" function "metrics" took 4.967609ms to process 0 records I0702 02:13:20.967744 1 tasks_processing.go:74] worker 13 stopped. I0702 02:13:20.967749 1 tasks_processing.go:71] worker 19 working on ingress task. I0702 02:13:20.969377 1 tasks_processing.go:74] worker 5 stopped. I0702 02:13:20.969633 1 recorder.go:70] Recording config/featuregate with fingerprint=40f746d0743e06a34c1b6e852c8c17ee0b2bdb4601f3d1d2c4be1af5bb66e53d I0702 02:13:20.969658 1 gather.go:181] gatherer "clusterconfig" function "feature_gates" took 15.216063ms to process 1 records I0702 02:13:20.970910 1 tasks_processing.go:74] worker 2 stopped. I0702 02:13:20.971198 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=ae027c31038bcbc5ff1f1d8d1fa29e173a85d66ded12a7f7643a96e810e80493 I0702 02:13:20.971259 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=dae4e3bdb2b623095ba39d2622d9db58193998a6c83c9acece78681c90abcbfe I0702 02:13:20.971298 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=348b04f3e6e6434857a4866e6b6c22d22758e95e1b807f24b6afc3bc3b9e4855 I0702 02:13:20.971312 1 gather.go:181] gatherer "clusterconfig" function "mutating_webhook_configurations" took 31.401435ms to process 3 records I0702 02:13:20.971404 1 recorder.go:70] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=ddb808e51d1b7da4b1bd86c0b4c3175002e74811e9a1428806bc03127a4ead7d I0702 02:13:20.971442 1 tasks_processing.go:74] worker 21 stopped. I0702 02:13:20.971500 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=02f1f85be99eeacce93c0b6b0df8694de741e1ea75c8332a08886c359d1039da I0702 02:13:20.971676 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=1c2736981d4e372ef062fff0d648c06d9742a46553dbe700e09cc6d33cddc2df I0702 02:13:20.971729 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=990b51c728a65384a81b5c294d2c9f7f280026495b49ccc77cb704d9cf12c9fe I0702 02:13:20.971806 1 recorder.go:70] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=1aec767fd9603ea4e7e124d9017b8b2da8c5aeffa8eec778e04099465834499b I0702 02:13:20.971849 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=9657e5fd1931a92778b4ec3552ac90b4ecf09abad25847365e79f7ea2e3c5892 I0702 02:13:20.971887 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=47c13142fbbec397b5a430e9bf1e202e7d060e5cd184502f4317f90f9c16ebb0 I0702 02:13:20.971923 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=a76db4fef8af0d4ffafc036a38f8dc063613434349a6890f86885584801d5429 I0702 02:13:20.971956 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=8727dea4baba6106701c38531797b6129974ddeba0ca5eaf1db282f79c28208f I0702 02:13:20.972034 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=72380a767de243bb7334e717e41e022758453d2e75ab873b8c9b83dd2122fb52 I0702 02:13:20.972092 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=bb90e742dfb8f1a582ab59666d8c5391bbf2e1bc6546acff05f8f0f888d05071 I0702 02:13:20.972137 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=2abcac51077220e79a17d4460975a91cfa3e3bbe44d82476ff67315af499b303 I0702 02:13:20.972174 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=761c1056757e2de967b965c0ecd725c787727733b3e74b5ac4d961d38aab7417 I0702 02:13:20.972210 1 gather.go:181] gatherer "clusterconfig" function "validating_webhook_configurations" took 31.346004ms to process 13 records E0702 02:13:20.972237 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:38530->172.30.0.10:53: read: connection refused I0702 02:13:20.972254 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0702 02:13:20.972262 1 gather.go:181] gatherer "clusterconfig" function "overlapping_namespace_uids" took 32.281489ms to process 1 records I0702 02:13:20.972312 1 tasks_processing.go:74] worker 4 stopped. I0702 02:13:20.972376 1 tasks_processing.go:74] worker 10 stopped. I0702 02:13:20.972454 1 recorder.go:70] Recording config/authentication with fingerprint=4612850898bbfe8fd193d705b9eb06cebb6bd52f3e813b3c73568ec06c69fc95 I0702 02:13:20.972469 1 gather.go:181] gatherer "clusterconfig" function "authentication" took 17.30055ms to process 1 records I0702 02:13:20.972541 1 recorder.go:70] Recording config/pdbs/openshift-console/console with fingerprint=c39f9801004f2625d9dd3979f63b8bf22d12fdfdfbea4c2f5b212c98f70514c2 I0702 02:13:20.972547 1 tasks_processing.go:74] worker 15 stopped. I0702 02:13:20.972564 1 recorder.go:70] Recording config/pdbs/openshift-console/downloads with fingerprint=ae22e3bd1a088819f1b4260bf106491b730f09fc059d5ba697bd4a969a6ea9ac I0702 02:13:20.972578 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=b911d76388cbb843dcc9304232baa9a6f855fd873ace53d4e1a9cf41c5b1953c I0702 02:13:20.972591 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=789881ef06726ba1e9285d7ef1c5a76f90edb649badbbab8f3e51937a749c646 I0702 02:13:20.972610 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=de4c7d34bc0ec0d383c68d7e92c65ee594881386e4f4a24df9058cd777350b23 I0702 02:13:20.972626 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=41e9a1b7912fa76291283a29ea911b50d4f4185266bfbb7d41ad6c8e2b1e3cff I0702 02:13:20.972641 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=645b6cf4a8597a913580e3406c8a8ef3a08bd86d5cbb8a306b8b32763e11ca67 I0702 02:13:20.972656 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=4b9d328b4e8e63476326b9cd78ee5b36e155bbd2e2ed9ddf651c21866ffb5832 I0702 02:13:20.972671 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=8caa6dd6569e7ec69e2273aaab757374be83198c77e7082fedab4265b7f69afc I0702 02:13:20.972687 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=782469968fce3bcab871fe40bd659af7e3891b38a46081ef5d6484e3ddfd1a9e I0702 02:13:20.972692 1 gather.go:181] gatherer "clusterconfig" function "pdbs" took 32.106016ms to process 10 records E0702 02:13:20.972717 1 gather.go:144] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.8:38530->172.30.0.10:53: read: connection refused I0702 02:13:20.972730 1 gather.go:181] gatherer "clusterconfig" function "active_alerts" took 7.320576ms to process 0 records I0702 02:13:20.972740 1 tasks_processing.go:74] worker 18 stopped. I0702 02:13:20.973282 1 tasks_processing.go:74] worker 28 stopped. I0702 02:13:20.973296 1 gather.go:181] gatherer "clusterconfig" function "certificate_signing_requests" took 34.700237ms to process 0 records I0702 02:13:20.975540 1 requests.go:204] Asking for SCA certificate for x86_64 architecture W0702 02:13:20.979032 1 sca.go:117] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:40261->172.30.0.10:53: read: connection refused I0702 02:13:20.979044 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:40261->172.30.0.10:53: read: connection refused I0702 02:13:20.980695 1 tasks_processing.go:74] worker 22 stopped. I0702 02:13:20.980710 1 gather.go:181] gatherer "clusterconfig" function "openshift_machine_api_events" took 22.173419ms to process 0 records I0702 02:13:20.980855 1 tasks_processing.go:74] worker 26 stopped. I0702 02:13:20.980866 1 gather.go:181] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 42.217782ms to process 0 records I0702 02:13:20.981246 1 tasks_processing.go:74] worker 25 stopped. I0702 02:13:20.981263 1 gather.go:181] gatherer "clusterconfig" function "jaegers" took 14.922807ms to process 0 records I0702 02:13:20.981515 1 tasks_processing.go:74] worker 1 stopped. I0702 02:13:20.981530 1 gather.go:181] gatherer "clusterconfig" function "node_logs" took 41.264258ms to process 0 records I0702 02:13:20.982029 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0702 02:13:20.982046 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0702 02:13:20.982053 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0702 02:13:20.982059 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0702 02:13:20.982073 1 tasks_processing.go:74] worker 3 stopped. I0702 02:13:20.982084 1 controller.go:457] The operator is still being initialized I0702 02:13:20.982092 1 controller.go:482] The operator is healthy I0702 02:13:20.982178 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=ee7828c09ba6459fc0514015935c42446f43a52cf79c9558c369c640355ef160 I0702 02:13:20.982202 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=8b85148bdc3f249252ead355d49500184ff71e748122550b6b00a88d239d763f I0702 02:13:20.982226 1 gather.go:181] gatherer "clusterconfig" function "storage_classes" took 43.036255ms to process 2 records I0702 02:13:20.982893 1 tasks_processing.go:74] worker 24 stopped. I0702 02:13:20.983609 1 recorder.go:70] Recording config/infrastructure with fingerprint=1b545c4c5e5a501ae6cd4d391181431b7b1690e661e02d5f763148431267b1b2 I0702 02:13:20.983629 1 gather.go:181] gatherer "clusterconfig" function "infrastructures" took 22.097958ms to process 1 records I0702 02:13:20.983831 1 tasks_processing.go:74] worker 16 stopped. I0702 02:13:20.984684 1 recorder.go:70] Recording config/node/ip-10-0-0-214.ec2.internal with fingerprint=a4936acaf6e7554c28c03b79d04abfef8ea33ce0e928b1aae859e81554193177 I0702 02:13:20.985011 1 recorder.go:70] Recording config/node/ip-10-0-1-108.ec2.internal with fingerprint=f6a3024a6f2c69d4f080dcccd44bf6499744989bd65e4c27b3a940557e9d7d71 I0702 02:13:20.985174 1 recorder.go:70] Recording config/node/ip-10-0-2-7.ec2.internal with fingerprint=f317fe16202bcb1fd48e28937b4a1fbae0cbcd1d7db8d99b333d8401cc2e962d I0702 02:13:20.985247 1 gather.go:181] gatherer "clusterconfig" function "nodes" took 29.431983ms to process 3 records W0702 02:13:20.987425 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:13:20.988536 1 tasks_processing.go:74] worker 23 stopped. I0702 02:13:20.988975 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=4b1933e116f937bfae649ca11cc080877618702e1f1180bcd402c24f76802141 I0702 02:13:20.989285 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=1308c700f6c9bffb507c52b98bd40c84a996c0f8fa313e83ae0da7e15c6d2566 I0702 02:13:20.989301 1 gather.go:181] gatherer "clusterconfig" function "clusterroles" took 36.856595ms to process 2 records I0702 02:13:20.989809 1 tasks_processing.go:74] worker 19 stopped. I0702 02:13:20.990037 1 recorder.go:70] Recording config/ingress with fingerprint=58c323d971acc2b2c5f5516fa4ff275fb7a1c4833127adf086de7480c6a5fa45 I0702 02:13:20.990059 1 gather.go:181] gatherer "clusterconfig" function "ingress" took 21.075551ms to process 1 records I0702 02:13:20.990116 1 tasks_processing.go:74] worker 9 stopped. I0702 02:13:20.990837 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=6b8dbda842ec0a9cd5d647cbc304e2e92e57a10e7c321609684b15c55f548bb0 I0702 02:13:20.991310 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=b4c4381aeb8a2ab1bae16d26a9b9fc7666f7abee168adbbfb128384c709dbd49 I0702 02:13:20.991327 1 gather.go:181] gatherer "clusterconfig" function "crds" took 51.687084ms to process 2 records I0702 02:13:20.996862 1 tasks_processing.go:74] worker 7 stopped. I0702 02:13:20.996996 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=c8dff9a1fe5d8591624465da4febdf962a9c985e94cb68e9d176850578998af9 I0702 02:13:20.997015 1 gather.go:181] gatherer "clusterconfig" function "schedulers" took 29.15792ms to process 1 records I0702 02:13:20.997112 1 tasks_processing.go:74] worker 31 stopped. I0702 02:13:20.997508 1 recorder.go:70] Recording config/version with fingerprint=eb1de04c773b617e057d8d15ed2c7e6dbfe6cfb3620733190acd0f664a1f1caa I0702 02:13:20.997522 1 recorder.go:70] Recording config/id with fingerprint=fcfdd22d80f31f6368fbf06b9381777551747c4175b2cdd8bb4eb43e7a902d03 I0702 02:13:20.997533 1 gather.go:181] gatherer "clusterconfig" function "version" took 51.553424ms to process 2 records I0702 02:13:20.998322 1 tasks_processing.go:74] worker 20 stopped. I0702 02:13:20.998529 1 recorder.go:70] Recording config/olm_operators with fingerprint=7117bbb4d6e7dfbd33fa430bbdfad2423ffaea19479aa06f0f0716a1f21701be I0702 02:13:20.998544 1 gather.go:181] gatherer "clusterconfig" function "olm_operators" took 32.717916ms to process 1 records I0702 02:13:21.000267 1 tasks_processing.go:74] worker 17 stopped. I0702 02:13:21.000290 1 gather.go:181] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 34.909139ms to process 0 records E0702 02:13:21.008495 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%!e(MISSING)f87c365-0973-4c43-99fb-f23dcd077e80%!+(MISSING)and+status+is+%!a(MISSING)ccepted%!"(MISSING): dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:42833->172.30.0.10:53: read: connection refused I0702 02:13:21.008518 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27ef87c365-0973-4c43-99fb-f23dcd077e80%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.8:42833->172.30.0.10:53: read: connection refused I0702 02:13:21.014114 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0702 02:13:21.014116 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0702 02:13:21.014114 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0702 02:13:21.015449 1 base_controller.go:73] Caches are synced for ConfigController I0702 02:13:21.015465 1 base_controller.go:110] Starting #1 worker of ConfigController controller ... I0702 02:13:21.016288 1 tasks_processing.go:74] worker 30 stopped. E0702 02:13:21.016309 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0702 02:13:21.016324 1 gather.go:144] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2r9nbgvmgvbn22v764b0rl5o6j84ga27-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2r9nbgvmgvbn22v764b0rl5o6j84ga27-primary-cert-bundle-secret" not found I0702 02:13:21.016425 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=dc4f1e074d29e850a5c4ca197a3e240cd36f53f8ab4e182a9edb74e6244fbea1 I0702 02:13:21.016452 1 gather.go:181] gatherer "clusterconfig" function "ingress_certificates" took 57.550913ms to process 1 records I0702 02:13:21.018342 1 tasks_processing.go:74] worker 8 stopped. I0702 02:13:21.019770 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z with fingerprint=7871d6d56829e2ec4f6294c1a795ed39e0857597aa7f7a1ea26b6d45be308f30 I0702 02:13:21.020003 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-fdwmw with fingerprint=45c16cfdab5c32bae3f03c5816d5a6e858180bf305ba61ee17af6d456135cb52 I0702 02:13:21.020092 1 recorder.go:70] Recording config/running_containers with fingerprint=d5a76aa3309c225fb1ec5ecc23cc12c8202b724547e0d3ea291c964759dd5dea I0702 02:13:21.020105 1 gather.go:181] gatherer "clusterconfig" function "container_images" took 78.324337ms to process 3 records I0702 02:13:21.023909 1 tasks_processing.go:74] worker 11 stopped. I0702 02:13:21.023922 1 gather.go:181] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 83.717114ms to process 0 records I0702 02:13:21.027872 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:13:21.028679 1 tasks_processing.go:74] worker 0 stopped. E0702 02:13:21.028692 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0702 02:13:21.028700 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0702 02:13:21.028704 1 gather.go:144] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0702 02:13:21.028741 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0702 02:13:21.028752 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0702 02:13:21.028758 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=e1a0ad594db3a5d5d50e9e25d7b235af20f84f5def41537f127febe525b834f2 I0702 02:13:21.028767 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0702 02:13:21.028806 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0702 02:13:21.028813 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0702 02:13:21.028818 1 gather.go:181] gatherer "clusterconfig" function "config_maps" took 74.686795ms to process 6 records I0702 02:13:21.045946 1 base_controller.go:73] Caches are synced for LoggingSyncer I0702 02:13:21.045965 1 base_controller.go:110] Starting #1 worker of LoggingSyncer controller ... I0702 02:13:21.067804 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress-operator" I0702 02:13:21.615028 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0702 02:13:21.814441 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" W0702 02:13:21.985572 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:13:22.193294 1 gather_cluster_operator_pods_and_events.go:121] Found 26 pods with 29 containers I0702 02:13:22.193313 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 867787 bytes I0702 02:13:22.193698 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-7d4ddc79d6-tv62z pod in namespace openshift-console-operator (previous: true). I0702 02:13:22.399206 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-7d4ddc79d6-tv62z pod in namespace openshift-console-operator (previous: false). I0702 02:13:22.602388 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-65b4bb4598-4frdc pod in namespace openshift-console (previous: false). I0702 02:13:22.615962 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0702 02:13:22.792798 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:22.792837 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-65b4bb4598-kvwff pod in namespace openshift-console (previous: false). W0702 02:13:22.986092 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:13:22.992709 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:22.992729 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-677967cb6f-fwfk8 pod in namespace openshift-console (previous: false). I0702 02:13:23.197053 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-7cd85464fd-hlwcv pod in namespace openshift-console (previous: false). I0702 02:13:23.219040 1 tasks_processing.go:74] worker 27 stopped. I0702 02:13:23.219243 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=b999c3a95a2e947c74006989f97ec344fe43121cf4b8286cfb74136052e85d10 I0702 02:13:23.219295 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=183539bbe89c989a326011f124cb26e91d4dc0406689baecde44742b1c4a8e80 I0702 02:13:23.219328 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=872cda535018584646efe33c14be3e29790b6e79c22587c966cc8cc257cfab2e I0702 02:13:23.219340 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0702 02:13:23.219363 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=02c02439d132acc69bea971fcc063e616156be278cab631b13810783765de0b0 I0702 02:13:23.219380 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0702 02:13:23.219408 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=70f329764d3869e4a421b54a902690bec6aeb52689ca6ab4c261b74588dbfcb1 I0702 02:13:23.219435 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=a4fb6dc9eae5bc4f2507a6488f5b949480aba31da44c37e90f6bf24c5c721e52 I0702 02:13:23.219462 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=3f3f4d1f8e38f6ba57e1fe825fddea467d1065f8279c66b238beb50c1030960a I0702 02:13:23.219500 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=7bf5b70ad21f3616263f289cb834106ba2544db9628f7bc001d64bc4f1377734 I0702 02:13:23.219837 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=6de874c32077d4603b814bf2752f8ed410eb3afdf93ea2be73a2fb6b473b3db0 I0702 02:13:23.219868 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=3d9cda40db5712d5b8e4acfc419aab28624c4a33b97ccf781c190f6b148b8c60 I0702 02:13:23.219878 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0702 02:13:23.219894 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=2842a26e3c3d1c232bbc0f67810b2ac139b3e3dd90e9b2cb851b80a05b1f360a I0702 02:13:23.219906 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0702 02:13:23.219921 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=146a76183829e34d071f48b96b6e89e54aba19ddf61fa58e9e84efa04342a7e8 I0702 02:13:23.219932 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0702 02:13:23.219955 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=0f1ee5c86f20561f7f1bab00cc6a6aef89f84b34d0f86ffd1271fe7e67cf40da I0702 02:13:23.219968 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0702 02:13:23.219988 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=ed1564cb101c202e7b65259d8d79443a77269f0a3c9873e5bdc307bfe52eb243 I0702 02:13:23.220108 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=1aa2b94e7d5d3b4a55e530036433db525c6b4b823bf1a5823d93c7942953e7d2 I0702 02:13:23.220141 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0702 02:13:23.220164 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=baf0462e79a11b5052aa9b018c225e17263db7d420e044ca246993384b4d9e4c I0702 02:13:23.220192 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=d1442b97f7d392d532fba031a965ab26169332616e20f015f4f78007ebf2111d I0702 02:13:23.220204 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0702 02:13:23.220242 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=5a52f620f4ef1f4a1b0eabe150541e79e9c1d7a52fe6d76e49b30268ba419b46 I0702 02:13:23.220256 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0702 02:13:23.220270 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=60bfde37e8dc6030e57a73cd1bef61f94a0f16a15f8bc07c214cd5574614fc3d I0702 02:13:23.220285 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=e5f35d3a10b185c636b8f1fcdde8eb8f50415a0f67481330c791691aeb22406e I0702 02:13:23.220301 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=cadebe92d77e400605f7e532189e1925e75b9a37bbbce6008e9d6819e330106e I0702 02:13:23.220323 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=90d2a1699730323b0ffc26480823783dc53f4df66a8a110bab3363aa838a053d I0702 02:13:23.220345 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=8142028e342deda8abe7194241de80cb1cfef07d6f95c7d381c22e918f51a374 I0702 02:13:23.220355 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0702 02:13:23.220380 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=feb29a06d4150ee0bc001766f9ac3f38fcbe9b46181ec5b64b9bedbf531aa111 I0702 02:13:23.220404 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0702 02:13:23.220414 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0702 02:13:23.220421 1 gather.go:181] gatherer "clusterconfig" function "operators" took 2.271268621s to process 36 records I0702 02:13:23.399354 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-bc8795f-j4cmj pod in namespace openshift-console (previous: false). I0702 02:13:23.601705 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-6d4c9b4b6c-d4vt2 pod in namespace openshift-console (previous: false). I0702 02:13:23.797726 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-6d4c9b4b6c-jqfkj pod in namespace openshift-console (previous: false). W0702 02:13:23.985836 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:13:23.998165 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-5bv98 pod in namespace openshift-dns (previous: false). I0702 02:13:24.197369 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-5bv98 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-5bv98\" is waiting to start: ContainerCreating" I0702 02:13:24.197385 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-5bv98\" is waiting to start: ContainerCreating" I0702 02:13:24.197395 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-5bv98 pod in namespace openshift-dns (previous: false). I0702 02:13:24.400908 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-5bv98 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-5bv98\" is waiting to start: ContainerCreating" I0702 02:13:24.400934 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-5bv98\" is waiting to start: ContainerCreating" I0702 02:13:24.400977 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-62ld7 pod in namespace openshift-dns (previous: false). I0702 02:13:24.600374 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-62ld7 pod in namespace openshift-dns (previous: false). I0702 02:13:24.803069 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-rc9z4 pod in namespace openshift-dns (previous: false). W0702 02:13:24.986034 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0702 02:13:24.997022 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-rc9z4 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-rc9z4\" is waiting to start: ContainerCreating" I0702 02:13:24.997041 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-rc9z4\" is waiting to start: ContainerCreating" I0702 02:13:24.997049 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-rc9z4 pod in namespace openshift-dns (previous: false). I0702 02:13:25.201773 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-rc9z4 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-rc9z4\" is waiting to start: ContainerCreating" I0702 02:13:25.201795 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-rc9z4\" is waiting to start: ContainerCreating" I0702 02:13:25.201810 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-98rq7 pod in namespace openshift-dns (previous: false). I0702 02:13:25.398062 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:25.398086 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-t9fln pod in namespace openshift-dns (previous: false). I0702 02:13:25.597910 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:25.597932 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-zbfdq pod in namespace openshift-dns (previous: false). I0702 02:13:25.798249 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:25.798269 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-76bc8b9f9-7lsm4 pod in namespace openshift-image-registry (previous: false). W0702 02:13:25.985879 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0702 02:13:25.985905 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0702 02:13:25.985923 1 tasks_processing.go:74] worker 12 stopped. E0702 02:13:25.985933 1 gather.go:144] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0702 02:13:25.985944 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0702 02:13:25.985955 1 gather.go:181] gatherer "clusterconfig" function "dvo_metrics" took 5.039791744s to process 1 records I0702 02:13:25.998303 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-76bc8b9f9-jsqzb pod in namespace openshift-image-registry (previous: false). I0702 02:13:26.199648 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-b66d5d98f-xghpj pod in namespace openshift-image-registry (previous: false). I0702 02:13:26.398895 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-4h8q8 pod in namespace openshift-image-registry (previous: false). I0702 02:13:26.598063 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:26.598083 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-j27bl pod in namespace openshift-image-registry (previous: false). I0702 02:13:26.801674 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:26.801693 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-qk9j7 pod in namespace openshift-image-registry (previous: false). I0702 02:13:26.998163 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0702 02:13:26.998183 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-788f75dd75-w7s7n pod in namespace openshift-ingress (previous: false). I0702 02:13:27.199579 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-78f674cd4c-kjk4j pod in namespace openshift-ingress (previous: false). I0702 02:13:27.398140 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-78f674cd4c-tmqmw pod in namespace openshift-ingress (previous: false). I0702 02:13:27.601383 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-hc4qf pod in namespace openshift-ingress-canary (previous: false). I0702 02:13:27.798036 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-mzhj9 pod in namespace openshift-ingress-canary (previous: false). I0702 02:13:27.997947 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-qnbs7 pod in namespace openshift-ingress-canary (previous: false). I0702 02:13:28.201835 1 tasks_processing.go:74] worker 29 stopped. I0702 02:13:28.202028 1 recorder.go:70] Recording events/openshift-console-operator with fingerprint=3f5609ef0dba1fc6cb8ca72ae52589a1439f56333a1943d88b2a6ea2da438220 I0702 02:13:28.202138 1 recorder.go:70] Recording events/openshift-console with fingerprint=5e51d26491e25364f20b4a22e7730d76e58a29601ca24848435c4d7335c7c3b8 I0702 02:13:28.202160 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=bc013b4a0ead83fcb87784ea320c87a5636505c90aa19ff6b9c269709239fab3 I0702 02:13:28.202206 1 recorder.go:70] Recording events/openshift-dns with fingerprint=94a35b2615046ac3137f838a105e3673795ed002d17675deabbbf5d17810ba1c I0702 02:13:28.202376 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=b66c4ee1fd7d57f17d474d6ee4a1096a4056f95ef319a376eab3c32c11167463 I0702 02:13:28.202401 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=3b5def573ccdf491fa073e690ae3ed3f79bc2c581d16599fcda1b78ba709f7e9 I0702 02:13:28.202480 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=3e1d6fc9dfdfe3011545807a232ba0a2e35d3993d73d1b26500e0ac864de094a I0702 02:13:28.202506 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=e575f426590c08e95fdd3fe57477d92331c1ecacf97bb5b3f5f4144c0600c820 I0702 02:13:28.202655 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z with fingerprint=7871d6d56829e2ec4f6294c1a795ed39e0857597aa7f7a1ea26b6d45be308f30 E0702 02:13:28.202678 1 gather.go:165] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json" was already recorded and had the fingerprint "7871d6d56829e2ec4f6294c1a795ed39e0857597aa7f7a1ea26b6d45be308f30", overwriting with the record having fingerprint "7871d6d56829e2ec4f6294c1a795ed39e0857597aa7f7a1ea26b6d45be308f30" W0702 02:13:28.202690 1 gather.go:159] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json" because of the warning: warning: the record with the same fingerprint "7871d6d56829e2ec4f6294c1a795ed39e0857597aa7f7a1ea26b6d45be308f30" was already recorded at path "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json" I0702 02:13:28.202715 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-7d4ddc79d6-tv62z/console-operator_previous.log with fingerprint=0dc47323d31efaf82f127821cb176745406af0a0993936b72ff48241f4b5b8e2 I0702 02:13:28.202934 1 recorder.go:70] Recording config/pod/openshift-console-operator/logs/console-operator-7d4ddc79d6-tv62z/console-operator_current.log with fingerprint=9605a45dba704671b5fb3fb153a5957173b126c4dee3d151ad65ae7747f31885 I0702 02:13:28.202945 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-677967cb6f-fwfk8/console_current.log with fingerprint=9f24677f4ade7dc390f68b58a9012ec25fd79043c0c5d6463844fdc2fc851d14 I0702 02:13:28.202955 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-7cd85464fd-hlwcv/console_current.log with fingerprint=79a54f22e6182c6960da627a0a19b951983909aecfb0743f117cf63e77a3e040 I0702 02:13:28.202965 1 recorder.go:70] Recording config/pod/openshift-console/logs/console-bc8795f-j4cmj/console_current.log with fingerprint=b30ac88bdb2db81d2904c27140e02b312d247b1d9bce374eda572de7bfdf436c I0702 02:13:28.202971 1 recorder.go:70] Recording config/pod/openshift-console/logs/downloads-6d4c9b4b6c-d4vt2/download-server_current.log with fingerprint=76acdc3a78c94fb35f8ad98cff97e0f217c61f27a577cb1f6e56baafba0357d0 I0702 02:13:28.202978 1 recorder.go:70] Recording config/pod/openshift-console/logs/downloads-6d4c9b4b6c-jqfkj/download-server_current.log with fingerprint=30715054c763067b6ed7ad1b36684174faee885a98d667b23e29773565fd161c I0702 02:13:28.203074 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-5bv98 with fingerprint=cb1c8d712d1f2e242dea4fd258a7af10885b14cd13ed7bb15ad69e59cfa0f87e I0702 02:13:28.203141 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-62ld7 with fingerprint=c84e0cd807724f1ed7b80ec0e36bdd8ce69d4156ee65b6a323214eb6c7b035b8 I0702 02:13:28.203150 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-62ld7/dns_current.log with fingerprint=f0f5de428006cfb5e012b32f90bc4b9cb0439745b2c985e529f66bd024620212 I0702 02:13:28.203157 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-62ld7/kube-rbac-proxy_current.log with fingerprint=f8a358b90ef60c17e0443901741be2f43be7b8c38198d0080bc0360df2cd4b34 I0702 02:13:28.203244 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-rc9z4 with fingerprint=2d3855ff6886c0ea2e5792980adb7ba5ae908faa36a38bb5430c2c98dc625765 I0702 02:13:28.203261 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-76bc8b9f9-7lsm4/registry_current.log with fingerprint=333ab7cad340984529e94f157b08f1410c82891c37c1693e5de6ccbefd73ac4d I0702 02:13:28.203289 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-76bc8b9f9-jsqzb/registry_current.log with fingerprint=1a3329eaba9c9327c6333bfda815f1d6c963ee14f3dc8a3d9eaaeca3dc564e8a I0702 02:13:28.203328 1 recorder.go:70] Recording config/pod/openshift-image-registry/logs/image-registry-b66d5d98f-xghpj/registry_current.log with fingerprint=bb08e0c51cd2c72dfca14a8154637ab8282211bcf686beccade77cf55716ec4f I0702 02:13:28.203344 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-788f75dd75-w7s7n/router_current.log with fingerprint=0e4043fdff56f41b223b492ebfca3394f3c7447c919730a042494b5acca6ee95 I0702 02:13:28.203361 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-78f674cd4c-kjk4j/router_current.log with fingerprint=20b5b1d967b4b7922f503f54510b2e9f403f2fb0cf489a0d65b7869f0cc48956 I0702 02:13:28.203377 1 recorder.go:70] Recording config/pod/openshift-ingress/logs/router-default-78f674cd4c-tmqmw/router_current.log with fingerprint=83d088d48645a575b2fd6f3e2a103932fef7aad6dcf6addcf7e6940bbd6ee29b I0702 02:13:28.203441 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-hc4qf with fingerprint=42511176fc52907d0bbf02d12527c3d4797517350a37a4b4417f3c6e5c4ffc59 I0702 02:13:28.203448 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-hc4qf/serve-healthcheck-canary_current.log with fingerprint=d93a025775594a99cc299efac90a8ea3bc4ece936fc275e1f370e461950b04c0 I0702 02:13:28.203500 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-mzhj9 with fingerprint=d2c706176c04c6fec06939a3939a4c2f162a4fd598289db0933ebbc623bf8ee2 I0702 02:13:28.203507 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-mzhj9/serve-healthcheck-canary_current.log with fingerprint=8a0fc801b4d8af6e7da5466bedc7be89c115ef15268f7e77eb3e4d3d63727a92 I0702 02:13:28.203560 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-qnbs7 with fingerprint=f22786a143e5dfbb59db03e1c771f705246f0ccdff49a82ecc85e396652ac618 I0702 02:13:28.203567 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-qnbs7/serve-healthcheck-canary_current.log with fingerprint=68c0175f37ece90f5a59e77359bf12d7a7ace9efacf8a713d99a744a2e3a4766 I0702 02:13:28.203573 1 gather.go:181] gatherer "clusterconfig" function "operators_pods_and_events" took 7.247687743s to process 33 records I0702 02:13:33.977640 1 tasks_processing.go:74] worker 14 stopped. I0702 02:13:33.977697 1 recorder.go:70] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0702 02:13:33.977714 1 gather.go:181] gatherer "clusterconfig" function "install_plans" took 13.038241183s to process 1 records I0702 02:13:34.772971 1 tasks_processing.go:74] worker 6 stopped. I0702 02:13:34.773305 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=d309c5d800188ed390d5318f4d495169a88f9d6a7a83f87c3c59208ec78fa56d I0702 02:13:34.773322 1 gather.go:181] gatherer "clusterconfig" function "service_accounts" took 13.807218984s to process 1 records E0702 02:13:34.773372 1 periodic.go:252] clusterconfig failed after 13.836s with: function "tsdb_status" failed with an error, function "machine_healthchecks" failed with an error, function "silenced_alerts" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_configs" failed with an error, function "support_secret" failed with an error, function "machines" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json" I0702 02:13:34.773386 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "tsdb_status" failed with an error, function "machine_healthchecks" failed with an error, function "silenced_alerts" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_configs" failed with an error, function "support_secret" failed with an error, function "machines" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-7d4ddc79d6-tv62z.json" I0702 02:13:34.773393 1 periodic.go:214] Running workloads gatherer I0702 02:13:34.773407 1 tasks_processing.go:45] number of workers: 2 I0702 02:13:34.773417 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 02:13:34.773421 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0702 02:13:34.773440 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 02:13:34.773462 1 tasks_processing.go:71] worker 0 working on workload_info task. I0702 02:13:34.805251 1 gather_workloads_info.go:281] Loaded pods in 0s, will wait 24s for image data I0702 02:13:34.813840 1 tasks_processing.go:74] worker 1 stopped. I0702 02:13:34.813860 1 gather.go:181] gatherer "workloads" function "helmchart_info" took 40.407049ms to process 0 records I0702 02:13:34.816410 1 gather_workloads_info.go:390] No image sha256:3a2c82b772ea6072e1eca0efc5303f002189e80b0e985fb3e3b8ea9a12fe8060 (13ms) I0702 02:13:34.827918 1 gather_workloads_info.go:390] No image sha256:586127c49e25052113263d8aeaf6a1623aef22987364f119a17b2848982ff716 (11ms) I0702 02:13:34.840942 1 gather_workloads_info.go:390] No image sha256:41029e57ece558b2e0b12174ce924647fd9d5c5d83aadf4bba03d6916d3f9f54 (13ms) I0702 02:13:34.862074 1 gather_workloads_info.go:390] No image sha256:0c1abef76d6d2832ddcff31e950ffafeb4e0f6fad80a5f5dca94591de0fc9581 (21ms) I0702 02:13:34.886027 1 gather_workloads_info.go:390] No image sha256:3fea7db586d3236c1876a8b523af0d8d65c84d6f6cf5d0f4e41cafcffe6db5b5 (24ms) I0702 02:13:34.899190 1 gather_workloads_info.go:390] No image sha256:788d16bfff4bd6bb3ce97acc643b56fae6e619674ab279535480e519b4d5b769 (13ms) I0702 02:13:34.911917 1 gather_workloads_info.go:390] No image sha256:351ca92471647f383f3cd7f0b70f33f5fa11c39ab569c4948dff7d471a6c8be3 (13ms) I0702 02:13:34.924994 1 gather_workloads_info.go:390] No image sha256:02283a64c1936a5dd09d5c6bc0ecbcf9196fe4df6a4055c8be22cebd850aa7a3 (13ms) I0702 02:13:34.944035 1 gather_workloads_info.go:390] No image sha256:5bcfb1b739fa4e6ff9e21e4843f8520f29608135341f7d8dcc31084b9ba25f1d (19ms) I0702 02:13:34.959896 1 gather_workloads_info.go:390] No image sha256:751bd7eb62f632c0eef0ca91b6c57c64d4f1279d4dbe1a9b875ac4020a7b693f (16ms) I0702 02:13:34.975991 1 gather_workloads_info.go:390] No image sha256:c7aa910ca06f8fd89cdf2519136e0f3533bdacfc6bd58a3f6683f9f08756c7b1 (16ms) I0702 02:13:35.013440 1 gather_workloads_info.go:390] No image sha256:1eee856bde9bad71c4444c494225a0cf02d1f4a5293acf0bd6e7e1814efa0064 (37ms) I0702 02:13:35.112879 1 gather_workloads_info.go:390] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (99ms) I0702 02:13:35.213499 1 gather_workloads_info.go:390] No image sha256:0242cefbf0bb876101c4d67ab4e07401cb0ba6c02be2b73a4f17ba33b2960005 (101ms) I0702 02:13:35.312820 1 gather_workloads_info.go:390] No image sha256:2897724cec17bc13acc133181f64ccb8d54ce5dc9582f9cecf704c2540e15600 (99ms) I0702 02:13:35.413158 1 gather_workloads_info.go:390] No image sha256:38787de16ee8c686305f93b501d891b50cde24136009863869d230c0d4eeee5c (100ms) I0702 02:13:35.513823 1 gather_workloads_info.go:390] No image sha256:06821dfe231330e0eb5be915b9468cd13826a7ee59d1a732b89cb140548eade2 (101ms) I0702 02:13:35.618793 1 gather_workloads_info.go:390] No image sha256:8e158c83ec12b3e0237fab9a221630bd691e79c079a111f346354ff43b8199df (105ms) I0702 02:13:35.713860 1 gather_workloads_info.go:390] No image sha256:09b3e9e6a4f382fb4bbdb67bc945bcf19d66f604b7c8ffd66d4d4a2c94224248 (95ms) I0702 02:13:35.813471 1 gather_workloads_info.go:390] No image sha256:b8781637560d49de9b8e8cdd59ec75f03d98921ea66950381849398128cc9cb9 (100ms) I0702 02:13:35.913599 1 gather_workloads_info.go:390] No image sha256:3969660e807e8ce5656a74e5816573656241946ef1d766a9d2e21d434356313c (100ms) I0702 02:13:36.014438 1 gather_workloads_info.go:390] No image sha256:bc829520eeec090b60d17abbad1a22b34ea99b426193395808a778d73a5fb57a (101ms) I0702 02:13:36.126028 1 gather_workloads_info.go:390] No image sha256:1c7c24111f1709d7648156873ced09462638aa4b3cf7e7aee13297e51bcdfb3e (112ms) I0702 02:13:36.213380 1 gather_workloads_info.go:390] No image sha256:70f4702845ec5b627d49504a53a13d997c0578630cddffb89aba5cfebe04fc25 (87ms) I0702 02:13:36.312749 1 gather_workloads_info.go:390] No image sha256:8562a483f4568254250f19fc68d79f88c94a48508aefe59a70c1021a3aad000a (99ms) I0702 02:13:36.412926 1 gather_workloads_info.go:390] No image sha256:31a22f6ee061d0f6558664e580ec0f80f1927d4ec29626cf3e9420837f5d80e5 (100ms) I0702 02:13:36.513501 1 gather_workloads_info.go:390] No image sha256:68abf16a3b6c33fdd22a823c7d00f96aaaa8015d370116962a5835984908645a (101ms) I0702 02:13:36.613563 1 gather_workloads_info.go:390] No image sha256:0158b297aec0a66376c04d3c7cf4828db5e5d9f424e331ff7c7b796828036875 (100ms) I0702 02:13:36.712966 1 gather_workloads_info.go:390] No image sha256:221622a101db9797df0622b4534e3e42c4498df458f355a8a02fad8d65f8e3f2 (99ms) I0702 02:13:36.814749 1 gather_workloads_info.go:390] No image sha256:0c6b7f828184b16a5e6630111e0715e11d34dae9b0bf64619466543e6168e05c (102ms) I0702 02:13:36.913058 1 gather_workloads_info.go:390] No image sha256:42b4c6bdf3081ec264bc5edbc4eb779151de9e4742d6fe59a316645891df649c (98ms) I0702 02:13:37.013653 1 gather_workloads_info.go:390] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (101ms) I0702 02:13:37.113170 1 gather_workloads_info.go:390] No image sha256:50076d29400ecf9b704d8175509fc8cb7c28bea91d46a28dc37cbda1668c69f9 (99ms) I0702 02:13:37.213164 1 gather_workloads_info.go:390] No image sha256:30ff5e5b72f274598704ca69f1832eba1040cc25da47386004fd59500b6e4c55 (100ms) I0702 02:13:37.314304 1 gather_workloads_info.go:390] No image sha256:c96b4bddf3dd7daaa797cbda4dd92e981de2f5fc3404c01381f31dc0414d5c8d (101ms) I0702 02:13:37.415509 1 gather_workloads_info.go:390] No image sha256:194ee1155b87e110bc6c4a6e55667e41f3bdc8292e26a9561f9082fdfe467eec (101ms) I0702 02:13:37.514022 1 gather_workloads_info.go:390] No image sha256:6a9b4fd887fa7f1729302fb0330c3c9a483197bf78becdcfda4235ec823a2c39 (98ms) I0702 02:13:37.613460 1 gather_workloads_info.go:390] No image sha256:24519dcc0e63cdc60a639dd474d06e7612e9b76966bc60322cd9026a0f1b878c (99ms) I0702 02:13:37.712973 1 gather_workloads_info.go:390] No image sha256:a7fc4d05c4a3c82c9f355ca985d6fd26033c0d6a98716f25fcd2f54bae1d2fa0 (99ms) I0702 02:13:37.813601 1 gather_workloads_info.go:390] No image sha256:4b9b7949d209e257a76053de4ac74193f747c0d42e25dcd270d655849a4e51aa (101ms) I0702 02:13:37.913173 1 gather_workloads_info.go:390] No image sha256:894452e6587de55d04e37163d3da53e11fced8fb08bd0acf61e6d49f7964284e (100ms) I0702 02:13:38.013343 1 gather_workloads_info.go:390] No image sha256:80fdd5049bd2815e49c198a795a116976a852e8de630be74872439ba6c08758a (100ms) I0702 02:13:38.113768 1 gather_workloads_info.go:390] No image sha256:ed0c9631d6e424912cd8ce78ca7a2c4f76b3fc15e9e34fe00999170ccf19a611 (100ms) I0702 02:13:38.213335 1 gather_workloads_info.go:390] No image sha256:48dff3fcb0da58eb9b2b500f89af8b7d0790ffa60da0b0e301feca971dbafcbb (100ms) I0702 02:13:38.314092 1 gather_workloads_info.go:390] No image sha256:13aabbaf260d634ff0e9ac1af88af075b5ba671fd51313e33e74d7d2f7e54128 (101ms) I0702 02:13:38.413356 1 gather_workloads_info.go:390] No image sha256:4dcdadc14b7bdb2c00954d92e957d63aafbbe5f4047fe028e575d65725923cb0 (99ms) I0702 02:13:38.413392 1 tasks_processing.go:74] worker 0 stopped. I0702 02:13:38.413757 1 recorder.go:70] Recording config/workload_info with fingerprint=3a60bae4d05e32b0c67d745decdb46511e5865e1e3f2c26b850357b486b2a57c I0702 02:13:38.413775 1 gather.go:181] gatherer "workloads" function "workload_info" took 3.639920978s to process 1 records I0702 02:13:38.413789 1 periodic.go:261] Periodic gather workloads completed in 3.64s I0702 02:13:38.413797 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0702 02:13:38.413802 1 periodic.go:214] Running conditional gatherer I0702 02:13:38.421706 1 requests.go:282] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.18.9/gathering_rules I0702 02:13:38.552164 1 conditional_gatherer.go:336] updating alerts cache for conditional gatherer E0702 02:13:38.554649 1 conditional_gatherer.go:322] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.115.196:9091: connect: connection refused I0702 02:13:38.554723 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0702 02:13:38.563349 1 conditional_gatherer.go:390] cluster version is '4.18.9' E0702 02:13:38.563367 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563373 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563376 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563379 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563382 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563386 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563389 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563391 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563395 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0702 02:13:38.563398 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0702 02:13:38.563412 1 tasks_processing.go:45] number of workers: 3 I0702 02:13:38.563421 1 tasks_processing.go:69] worker 2 listening for tasks. I0702 02:13:38.563426 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0702 02:13:38.563434 1 tasks_processing.go:69] worker 0 listening for tasks. I0702 02:13:38.563447 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0702 02:13:38.563448 1 tasks_processing.go:69] worker 1 listening for tasks. I0702 02:13:38.563453 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0702 02:13:38.563458 1 tasks_processing.go:74] worker 1 stopped. I0702 02:13:38.563526 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0702 02:13:38.563538 1 gather.go:181] gatherer "conditional" function "conditional_gatherer_rules" took 1.168µs to process 1 records I0702 02:13:38.563572 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0702 02:13:38.563581 1 gather.go:181] gatherer "conditional" function "remote_configuration" took 1.278µs to process 1 records I0702 02:13:38.563587 1 tasks_processing.go:74] worker 0 stopped. I0702 02:13:38.563668 1 tasks_processing.go:74] worker 2 stopped. I0702 02:13:38.563680 1 gather.go:181] gatherer "conditional" function "rapid_container_logs" took 207.829µs to process 0 records I0702 02:13:38.563695 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=AsExpected message= I0702 02:13:38.563711 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f W0702 02:13:38.590934 1 gather.go:213] can't read cgroups memory usage data: open /sys/fs/cgroup/memory/memory.usage_in_bytes: no such file or directory I0702 02:13:38.591145 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=bab274789c2b4b4946247bac1edabf1be6f7069c52a458a4c6e6464185a531b2 I0702 02:13:38.591332 1 diskrecorder.go:70] Writing 137 records to /var/lib/insights-operator/insights-2026-07-02-021338.tar.gz I0702 02:13:38.601693 1 diskrecorder.go:51] Wrote 137 records to disk in 10ms I0702 02:13:38.601743 1 periodic.go:283] Gathering cluster info every 2h0m0s I0702 02:13:38.601770 1 periodic.go:284] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0702 02:14:50.937355 1 diskrecorder.go:170] Found files to send: insights-2026-07-02-021338.tar.gz I0702 02:14:50.937383 1 insightsuploader.go:150] Checking archives to upload periodically every 15m46.526927796s I0702 02:14:50.937392 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0702 02:14:50.952174 1 requests.go:47] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0702 02:14:51.137065 1 requests.go:88] Successfully reported id=2026-07-02T02:14:50Z x-rh-insights-request-id=a8e77f2660244e438dac3203319a3de9, wrote=77543 I0702 02:14:51.137138 1 insightsuploader.go:187] Uploaded report successfully in 199.737347ms I0702 02:14:51.137166 1 controller.go:119] Initializing last reported time to 2026-07-02T02:14:50Z I0702 02:14:51.137255 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0702 02:14:51.137268 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0702 02:14:51.137277 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0702 02:14:51.144346 1 controller.go:482] The operator is healthy I0702 02:15:20.951338 1 controller.go:482] The operator is healthy I0702 02:15:52.378203 1 insightsreport.go:137] Pulling report from smart-proxy I0702 02:15:52.378252 1 insightsreport.go:149] Retrieving report I0702 02:15:52.392514 1 requests.go:111] Retrieving report for cluster: ef87c365-0973-4c43-99fb-f23dcd077e80 I0702 02:15:52.392528 1 requests.go:112] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/ef87c365-0973-4c43-99fb-f23dcd077e80/reports I0702 02:15:52.395514 1 requests.go:122] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/ef87c365-0973-4c43-99fb-f23dcd077e80/reports I0702 02:15:52.644877 1 insightsreport.go:184] Report retrieved I0702 02:15:52.688264 1 insightsreport.go:239] Report retrieved correctly I0702 02:17:20.960449 1 controller.go:482] The operator is healthy I0702 02:18:20.936005 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:18:20.945429 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:18:20.945465 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:18:20.952108 1 secretconfigobserver.go:119] support secret does not exist I0702 02:19:20.952745 1 controller.go:482] The operator is healthy I0702 02:21:20.955572 1 controller.go:482] The operator is healthy I0702 02:23:20.937706 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:23:20.942450 1 configmapobserver.go:84] configmaps "insights-config" not found I0702 02:23:20.950843 1 controller.go:482] The operator is healthy I0702 02:23:20.952659 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:23:20.956974 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:23:20.956989 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:23:20.961430 1 secretconfigobserver.go:119] support secret does not exist I0702 02:25:20.957073 1 controller.go:482] The operator is healthy I0702 02:27:20.954850 1 controller.go:482] The operator is healthy I0702 02:28:20.961965 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0702 02:28:20.971827 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0702 02:28:20.971851 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0702 02:28:20.975834 1 secretconfigobserver.go:119] support secret does not exist I0702 02:29:20.953874 1 controller.go:482] The operator is healthy I0702 02:30:42.665758 1 insightsuploader.go:146] Nothing to report since 2026-07-02T02:14:50Z I0702 02:31:20.954608 1 controller.go:482] The operator is healthy