I0704 22:23:24.195553 1 cmd.go:253] Using service-serving-cert provided certificates I0704 22:23:24.197143 1 observer_polling.go:159] Starting file observer W0704 22:23:24.223533 1 builder.go:272] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-5ff5cb4f99" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0704 22:23:24.431620 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0704 22:23:24.431864 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0704 22:23:24.432434 1 secure_serving.go:57] Forcing use of http/1.1 only W0704 22:23:24.432461 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0704 22:23:24.432469 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0704 22:23:24.432475 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0704 22:23:24.432480 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0704 22:23:24.432491 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0704 22:23:24.432498 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0704 22:23:24.432529 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0704 22:23:24.438203 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes CPMSMachineNamePrefix ChunkSizeMiB CloudDualStackNodeIPs ConsolePluginContentSecurityPolicy DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags GatewayAPI GatewayAPIController HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy OnClusterBuild PersistentIPsForVirtualization PrivateHostedZoneAWS RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController VSphereDriverConfiguration VSphereMultiVCenters ValidatingAdmissionPolicy AWSClusterHostedDNS AutomatedEtcdBackup BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNS GCPCustomAPIEndpoints HighlyAvailableArbiter ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements SELinuxChangePolicy SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerification SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0704 22:23:24.438229 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ChunkSizeMiB", "CloudDualStackNodeIPs", "ConsolePluginContentSecurityPolicy", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "GatewayAPI", "GatewayAPIController", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "OnClusterBuild", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "VSphereDriverConfiguration", "VSphereMultiVCenters", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AutomatedEtcdBackup", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "HighlyAvailableArbiter", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "SELinuxChangePolicy", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0704 22:23:24.440820 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0704 22:23:24.440858 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0704 22:23:24.440852 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0704 22:23:24.440858 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0704 22:23:24.440891 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 22:23:24.440898 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 22:23:24.441144 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0704 22:23:24.441211 1 secure_serving.go:213] Serving securely on [::]:8443 I0704 22:23:24.441249 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" E0704 22:23:24.443469 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.18bf36698da9bd5d openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdditionalRoutingCapabilities\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"CPMSMachineNamePrefix\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"ConsolePluginContentSecurityPolicy\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"GatewayAPI\", \"GatewayAPIController\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MetricsCollectionProfiles\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"OnClusterBuild\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SetEIPForNLBIngressController\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ClusterVersionOperatorConfiguration\", \"DNSNameResolver\", \"DualReplica\", \"DyanmicServiceEndpointIBMCloud\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"Example2\", \"ExternalOIDCWithUIDAndExtraClaimMappings\", \"GCPClusterHostedDNS\", \"GCPCustomAPIEndpoints\", \"HighlyAvailableArbiter\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"KMSEncryptionProvider\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NewOLMCatalogdAPIV1Metas\", \"NewOLMOwnSingleNamespace\", \"NewOLMPreflightPermissionChecks\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"SELinuxChangePolicy\", \"SELinuxMount\", \"ShortCertRotation\", \"SignatureStores\", \"SigstoreImageVerification\", \"SigstoreImageVerificationPKI\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereConfigurableMaxAllowedBlockVolumesPerNode\", \"VSphereHostVMGroupZonal\", \"VSphereMultiDisk\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-07-04 22:23:24.438183261 +0000 UTC m=+0.282437635,LastTimestamp:2026-07-04 22:23:24.438183261 +0000 UTC m=+0.282437635,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W0704 22:23:24.446623 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0704 22:23:24.446650 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 22:23:24.446689 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0704 22:23:24.454432 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:23:24.454456 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0704 22:23:24.461870 1 secretconfigobserver.go:119] support secret does not exist I0704 22:23:24.468910 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:23:24.475843 1 secretconfigobserver.go:119] support secret does not exist I0704 22:23:24.479673 1 recorder.go:156] Pruning old reports every 5h25m31s, max age is 288h0m0s I0704 22:23:24.487672 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0704 22:23:24.487688 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0704 22:23:24.487697 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0704 22:23:24.487704 1 periodic.go:212] Running clusterconfig gatherer I0704 22:23:24.487710 1 insightsreport.go:296] Starting report retriever I0704 22:23:24.487720 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0704 22:23:24.487760 1 tasks_processing.go:45] number of workers: 32 I0704 22:23:24.487795 1 tasks_processing.go:69] worker 7 listening for tasks. I0704 22:23:24.487804 1 tasks_processing.go:69] worker 19 listening for tasks. I0704 22:23:24.487807 1 tasks_processing.go:71] worker 7 working on nodes task. I0704 22:23:24.487808 1 tasks_processing.go:69] worker 8 listening for tasks. I0704 22:23:24.487812 1 tasks_processing.go:69] worker 20 listening for tasks. I0704 22:23:24.487816 1 tasks_processing.go:69] worker 21 listening for tasks. I0704 22:23:24.487818 1 tasks_processing.go:69] worker 13 listening for tasks. I0704 22:23:24.487822 1 tasks_processing.go:69] worker 14 listening for tasks. I0704 22:23:24.487824 1 tasks_processing.go:69] worker 27 listening for tasks. I0704 22:23:24.487829 1 tasks_processing.go:69] worker 26 listening for tasks. I0704 22:23:24.487829 1 tasks_processing.go:69] worker 9 listening for tasks. I0704 22:23:24.487834 1 tasks_processing.go:69] worker 12 listening for tasks. I0704 22:23:24.487830 1 tasks_processing.go:69] worker 28 listening for tasks. I0704 22:23:24.487837 1 tasks_processing.go:69] worker 15 listening for tasks. I0704 22:23:24.487845 1 tasks_processing.go:69] worker 30 listening for tasks. I0704 22:23:24.487838 1 tasks_processing.go:69] worker 29 listening for tasks. I0704 22:23:24.487851 1 tasks_processing.go:69] worker 23 listening for tasks. I0704 22:23:24.487844 1 tasks_processing.go:69] worker 22 listening for tasks. I0704 22:23:24.487854 1 tasks_processing.go:69] worker 17 listening for tasks. I0704 22:23:24.487851 1 tasks_processing.go:69] worker 10 listening for tasks. I0704 22:23:24.487862 1 tasks_processing.go:69] worker 6 listening for tasks. I0704 22:23:24.487843 1 tasks_processing.go:69] worker 11 listening for tasks. I0704 22:23:24.487865 1 tasks_processing.go:69] worker 24 listening for tasks. I0704 22:23:24.487868 1 tasks_processing.go:69] worker 5 listening for tasks. I0704 22:23:24.487874 1 tasks_processing.go:71] worker 11 working on metrics task. I0704 22:23:24.487846 1 tasks_processing.go:69] worker 16 listening for tasks. I0704 22:23:24.487881 1 tasks_processing.go:71] worker 8 working on validating_webhook_configurations task. I0704 22:23:24.487883 1 tasks_processing.go:71] worker 5 working on authentication task. I0704 22:23:24.487882 1 tasks_processing.go:69] worker 3 listening for tasks. I0704 22:23:24.487884 1 tasks_processing.go:69] worker 25 listening for tasks. I0704 22:23:24.487890 1 tasks_processing.go:71] worker 28 working on dvo_metrics task. I0704 22:23:24.487898 1 tasks_processing.go:71] worker 27 working on service_accounts task. I0704 22:23:24.487897 1 tasks_processing.go:71] worker 25 working on openstack_controlplanes task. I0704 22:23:24.487885 1 tasks_processing.go:71] worker 16 working on sap_pods task. I0704 22:23:24.487976 1 tasks_processing.go:71] worker 12 working on storage_cluster task. I0704 22:23:24.487982 1 tasks_processing.go:71] worker 22 working on schedulers task. I0704 22:23:24.487984 1 tasks_processing.go:71] worker 6 working on sap_config task. I0704 22:23:24.487892 1 tasks_processing.go:71] worker 3 working on infrastructures task. I0704 22:23:24.487860 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 22:23:24.489458 1 tasks_processing.go:71] worker 0 working on pod_network_connectivity_checks task. I0704 22:23:24.487796 1 tasks_processing.go:69] worker 31 listening for tasks. I0704 22:23:24.487867 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 22:23:24.487872 1 tasks_processing.go:71] worker 20 working on operators task. I0704 22:23:24.487875 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 22:23:24.487875 1 tasks_processing.go:71] worker 24 working on support_secret task. I0704 22:23:24.487876 1 tasks_processing.go:71] worker 19 working on image_registries task. I0704 22:23:24.487853 1 tasks_processing.go:69] worker 4 listening for tasks. I0704 22:23:24.487953 1 tasks_processing.go:71] worker 23 working on feature_gates task. I0704 22:23:24.487957 1 tasks_processing.go:71] worker 15 working on cost_management_metrics_configs task. I0704 22:23:24.487962 1 tasks_processing.go:71] worker 21 working on ingress_certificates task. I0704 22:23:24.487963 1 tasks_processing.go:71] worker 30 working on openshift_machine_api_events task. I0704 22:23:24.487971 1 tasks_processing.go:71] worker 9 working on machine_healthchecks task. I0704 22:23:24.487966 1 tasks_processing.go:71] worker 29 working on nodenetworkconfigurationpolicies task. I0704 22:23:24.487966 1 tasks_processing.go:71] worker 13 working on machine_config_pools task. I0704 22:23:24.487971 1 tasks_processing.go:71] worker 14 working on clusterroles task. I0704 22:23:24.487978 1 tasks_processing.go:71] worker 17 working on lokistack task. I0704 22:23:24.487974 1 tasks_processing.go:71] worker 26 working on version task. I0704 22:23:24.487981 1 tasks_processing.go:71] worker 10 working on crds task. I0704 22:23:24.487799 1 tasks_processing.go:69] worker 18 listening for tasks. I0704 22:23:24.489789 1 tasks_processing.go:71] worker 1 working on openstack_dataplanenodesets task. I0704 22:23:24.490562 1 tasks_processing.go:71] worker 4 working on olm_operators task. I0704 22:23:24.489844 1 tasks_processing.go:71] worker 2 working on cluster_apiserver task. I0704 22:23:24.491187 1 tasks_processing.go:71] worker 18 working on tsdb_status task. I0704 22:23:24.489872 1 tasks_processing.go:71] worker 31 working on config_maps task. I0704 22:23:24.492730 1 tasks_processing.go:71] worker 12 working on operators_pods_and_events task. I0704 22:23:24.492742 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 4.731741ms to process 0 records I0704 22:23:24.493360 1 tasks_processing.go:71] worker 6 working on image_pruners task. I0704 22:23:24.493375 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 5.357287ms to process 0 records I0704 22:23:24.494975 1 tasks_processing.go:71] worker 16 working on machine_autoscalers task. I0704 22:23:24.494982 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 6.982338ms to process 0 records I0704 22:23:24.495511 1 tasks_processing.go:71] worker 0 working on pdbs task. E0704 22:23:24.495521 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0704 22:23:24.495541 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 5.908112ms to process 0 records E0704 22:23:24.497088 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.150.244:9091: connect: connection refused I0704 22:23:24.497114 1 tasks_processing.go:71] worker 18 working on monitoring_persistent_volumes task. E0704 22:23:24.497112 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.150.244:9091: connect: connection refused E0704 22:23:24.497127 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.150.244:9091: connect: connection refused I0704 22:23:24.497146 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 5.878435ms to process 0 records E0704 22:23:24.497164 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.150.244:9091: connect: connection refused I0704 22:23:24.497188 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 9.252367ms to process 0 records I0704 22:23:24.497207 1 tasks_processing.go:71] worker 11 working on ceph_cluster task. I0704 22:23:24.502859 1 tasks_processing.go:71] worker 25 working on image task. I0704 22:23:24.502871 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 14.949713ms to process 0 records I0704 22:23:24.504582 1 tasks_processing.go:71] worker 29 working on oauths task. I0704 22:23:24.504595 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 14.577777ms to process 0 records I0704 22:23:24.504614 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 14.69058ms to process 0 records E0704 22:23:24.504623 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0704 22:23:24.504632 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 14.368748ms to process 0 records I0704 22:23:24.504643 1 tasks_processing.go:71] worker 9 working on ingress task. I0704 22:23:24.504655 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 14.603279ms to process 0 records I0704 22:23:24.504664 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 13.72994ms to process 0 records I0704 22:23:24.504667 1 tasks_processing.go:71] worker 15 working on machine_sets task. I0704 22:23:24.504672 1 tasks_processing.go:71] worker 13 working on proxies task. I0704 22:23:24.504716 1 tasks_processing.go:71] worker 1 working on container_runtime_configs task. I0704 22:23:24.504900 1 tasks_processing.go:71] worker 3 working on nodenetworkstates task. I0704 22:23:24.505583 1 recorder.go:70] Recording config/infrastructure with fingerprint=7e4a17e2e747edf92959963d0aae042011f3c6d83e59b024c48c9d90ab864834 I0704 22:23:24.505599 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 16.807952ms to process 1 records I0704 22:23:24.505793 1 tasks_processing.go:71] worker 11 working on machine_configs task. I0704 22:23:24.505806 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 8.576153ms to process 0 records I0704 22:23:24.506061 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0704 22:23:24.506115 1 controller.go:317] The initial operator extension status is healthy I0704 22:23:24.506127 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0704 22:23:24.506133 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0704 22:23:24.506140 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 22:23:24.506160 1 controller.go:458] The operator is still being initialized I0704 22:23:24.506167 1 controller.go:481] The operator is healthy I0704 22:23:24.506177 1 tasks_processing.go:71] worker 24 working on node_logs task. E0704 22:23:24.506242 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0704 22:23:24.506269 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0704 22:23:24.506374 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0704 22:23:24.506281 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 15.970739ms to process 0 records I0704 22:23:24.506588 1 tasks_processing.go:71] worker 5 working on openstack_version task. I0704 22:23:24.506629 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer W0704 22:23:24.506608 1 operator.go:287] started I0704 22:23:24.507075 1 recorder.go:70] Recording config/authentication with fingerprint=b5a82cca768e593c4fd97ac8f1a10ba445a8ea203f8b3ca7afe628d11a2dcf21 I0704 22:23:24.507097 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 18.346255ms to process 1 records I0704 22:23:24.507181 1 tasks_processing.go:71] worker 22 working on active_alerts task. I0704 22:23:24.507183 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=1943775f77bc42b1e7e4db4b3401743ab65b1919014b8ac86aaa76c225da9084 I0704 22:23:24.507195 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 18.325249ms to process 1 records I0704 22:23:24.507201 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 11.670464ms to process 0 records I0704 22:23:24.507207 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 15.967718ms to process 0 records I0704 22:23:24.507292 1 tasks_processing.go:71] worker 17 working on openshift_logging task. I0704 22:23:24.507296 1 tasks_processing.go:71] worker 2 working on overlapping_namespace_uids task. I0704 22:23:24.507446 1 recorder.go:70] Recording config/apiserver with fingerprint=ac583a0bfeae0102d1932e8f37d038f81c5db456fffbd20a045c2b75c747dcd0 I0704 22:23:24.507458 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 15.752629ms to process 1 records I0704 22:23:24.507469 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 16.610566ms to process 0 records I0704 22:23:24.507480 1 tasks_processing.go:71] worker 30 working on container_images task. I0704 22:23:24.507299 1 tasks_processing.go:71] worker 16 working on machines task. I0704 22:23:24.507658 1 tasks_processing.go:71] worker 7 working on certificate_signing_requests task. I0704 22:23:24.507825 1 recorder.go:70] Recording config/node/ip-10-0-0-90.ec2.internal with fingerprint=b2f649b16369de33b8c20e69e9ef72d84211c17f9b105695453ef46571a8d115 I0704 22:23:24.507920 1 recorder.go:70] Recording config/node/ip-10-0-1-120.ec2.internal with fingerprint=924280db147f7243edc13786410aa994898f3dbdd02fbf848edd3a9d9cc62628 I0704 22:23:24.508001 1 recorder.go:70] Recording config/node/ip-10-0-2-131.ec2.internal with fingerprint=aff83022a23702a7eef0862bf0488fdb4fb96d1baa687d8e475921974b494e0a I0704 22:23:24.508032 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 19.478139ms to process 3 records I0704 22:23:24.508118 1 tasks_processing.go:71] worker 23 working on jaegers task. I0704 22:23:24.508144 1 recorder.go:70] Recording config/featuregate with fingerprint=40283d0b73f691158a070936a3c373f93dfe65ca6de749ecae3b4ad1280f46a4 I0704 22:23:24.508155 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 16.903073ms to process 1 records I0704 22:23:24.508422 1 tasks_processing.go:71] worker 8 working on networks task. I0704 22:23:24.508595 1 recorder.go:70] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=44471e0b59d05694b0a97b09f95cc7d615e98189355641db2a75a284d900e01e I0704 22:23:24.508721 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=348eb0860679c8409bdfaea5fcb3e920205ef4eb26faf7ba29f5d1c8f4b90b81 I0704 22:23:24.508882 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=70cecc8c74c0371f4bf81b282f3e2af1a071fbdea1f62cf3eeb23154e2c52027 I0704 22:23:24.508921 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=8b3e65babb5d216db75cb7013653b7195e5116d72d985324cee69844d93a6d6f I0704 22:23:24.508958 1 recorder.go:70] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=97660e3281fbd1b1ec7caf503c6ddb2b3eb70401bd690d388ed09dc44db411c2 I0704 22:23:24.509002 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=390f9f335f2db49fcbe340cbc1df997c3d939a0c377e749475a8432ff8ee0495 I0704 22:23:24.509045 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=8f2e3c37b142d756d67bd4d2f1bb482f3aaa8e1beb5202a050bae09fec843b60 I0704 22:23:24.509085 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=77c34f9599b8bd415cfae1ab20d8f95db2d8c5f93425f353b176a62ab26505c6 I0704 22:23:24.509130 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=ae9a09b491cb22d09a723a5fca372b0d3b5d13ce527e74eaffbb557e859ea12f I0704 22:23:24.509181 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=34a1d97895100751bc116d78c6994584dfd41b1b5e0a2c8a224feec2ea476a65 I0704 22:23:24.509222 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=d203dedb4a424ff14ebb1108866609fcac2f7bf02b425bfacc264944b6cc4180 I0704 22:23:24.509271 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=ad9bbd95d2fd2422d47a91406429384abf0c47e7dffee7ec4b3088114a503e97 I0704 22:23:24.509312 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=3a746ec2cf41bedb6f5a9eac83ef3afd8804b0c2a6fb11ff6d1cd2950d243f12 I0704 22:23:24.509325 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 20.527174ms to process 13 records I0704 22:23:24.515580 1 tasks_processing.go:71] worker 28 working on silenced_alerts task. I0704 22:23:24.515591 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0704 22:23:24.515601 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 27.674545ms to process 1 records I0704 22:23:24.517990 1 tasks_processing.go:71] worker 6 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0704 22:23:24.518632 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=9df936f2c97bfe319c54080efc0ef8f1b90593254a20983d8003d81b4b6258b0 I0704 22:23:24.518693 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 24.614855ms to process 1 records I0704 22:23:24.519815 1 tasks_processing.go:71] worker 25 working on aggregated_monitoring_cr_names task. I0704 22:23:24.520139 1 recorder.go:70] Recording config/image with fingerprint=4f10d72757d3f9ef595b51c724d3b7e8f79febf236747296216c3b11e09647c1 I0704 22:23:24.520160 1 gather.go:177] gatherer "clusterconfig" function "image" took 16.939717ms to process 1 records I0704 22:23:24.520444 1 tasks_processing.go:71] worker 19 working on storage_classes task. I0704 22:23:24.520848 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=9fc0b6c05cabaa224e64fa1cc5abfa41e0bbc26c71d0df1339e36a6889191e42 I0704 22:23:24.520868 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 29.973141ms to process 1 records I0704 22:23:24.532291 1 tasks_processing.go:71] worker 1 working on mutating_webhook_configurations task. I0704 22:23:24.532306 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 27.555423ms to process 0 records I0704 22:23:24.532378 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 27.391259ms to process 0 records I0704 22:23:24.532416 1 tasks_processing.go:71] worker 3 working on openstack_dataplanedeployments task. I0704 22:23:24.532980 1 tasks_processing.go:71] worker 23 working on sap_datahubs task. I0704 22:23:24.532990 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 24.845996ms to process 0 records I0704 22:23:24.535472 1 tasks_processing.go:71] worker 15 working on install_plans task. I0704 22:23:24.535490 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 30.793667ms to process 0 records I0704 22:23:24.535610 1 tasks_processing.go:74] worker 8 stopped. I0704 22:23:24.535839 1 recorder.go:70] Recording config/network with fingerprint=8426e67895a3789a1a4f14d330faf13b83ea83a014db284e6849aaacfc0c3332 I0704 22:23:24.535854 1 gather.go:177] gatherer "clusterconfig" function "networks" took 27.173358ms to process 1 records I0704 22:23:24.535927 1 recorder.go:70] Recording config/proxy with fingerprint=1c5917d31aa7fc437594045ffb549a1c65540c800dbdf136cb27c8a901dff258 I0704 22:23:24.535945 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 31.085208ms to process 1 records I0704 22:23:24.535957 1 tasks_processing.go:74] worker 13 stopped. I0704 22:23:24.536073 1 tasks_processing.go:74] worker 9 stopped. I0704 22:23:24.536207 1 recorder.go:70] Recording config/ingress with fingerprint=0dc4780f0b8bb651799e3575d19b1127a76a6b597f7687da4425691dc97e4617 I0704 22:23:24.536229 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 31.415268ms to process 1 records I0704 22:23:24.536324 1 tasks_processing.go:74] worker 29 stopped. I0704 22:23:24.536513 1 recorder.go:70] Recording config/oauth with fingerprint=eeda301cfffa07a172763a79ae7c514139e91824e96a50c8c6a0dbca0d73fa2a I0704 22:23:24.536526 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 31.470498ms to process 1 records I0704 22:23:24.536684 1 tasks_processing.go:74] worker 5 stopped. I0704 22:23:24.536700 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 30.060781ms to process 0 records I0704 22:23:24.536739 1 tasks_processing.go:74] worker 24 stopped. I0704 22:23:24.536753 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 30.54904ms to process 0 records I0704 22:23:24.537026 1 tasks_processing.go:74] worker 17 stopped. I0704 22:23:24.537041 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 29.676154ms to process 0 records I0704 22:23:24.537166 1 tasks_processing.go:74] worker 16 stopped. E0704 22:23:24.537177 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0704 22:23:24.537184 1 gather.go:177] gatherer "clusterconfig" function "machines" took 29.514485ms to process 0 records I0704 22:23:24.538276 1 tasks_processing.go:74] worker 14 stopped. I0704 22:23:24.538577 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=8ed0c0bcd9ad23c910f12eb67cc5e10dbdff19f80a17bc24f006037c0b608163 I0704 22:23:24.538983 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=d77e916d6235d37abcbeeb08d9fb48ce8760d004cdab34ab1386fe11a6a8899f I0704 22:23:24.539004 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 47.327511ms to process 2 records I0704 22:23:24.539173 1 recorder.go:70] Recording config/pdbs/openshift-console/console with fingerprint=02f082732ad3c01bca13270a949829b0ddf314f31f02ea3ab255d97e9ff38486 I0704 22:23:24.539223 1 tasks_processing.go:74] worker 0 stopped. I0704 22:23:24.539256 1 recorder.go:70] Recording config/pdbs/openshift-console/downloads with fingerprint=fa4a59e8658beb0f500ac17996c99968c86c1f547b4ec4c610678f1ee3c3afdf I0704 22:23:24.539349 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=e83c5242ab6e62cec90858c717d8390082f2fc3b9372f3e0c63f62235229af11 I0704 22:23:24.539444 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=c54923388ae20fb542ef0294d064f5f2140470e4b8846e8c16982af5cb2fd7d5 I0704 22:23:24.539533 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=6f5bdaf18fd078e664a4a5184faa68c6777661c9add363a74580bebf63004a5e I0704 22:23:24.539607 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=ba7a04d7dd35abc5aa22da98a291fa353042864dc698156cd6d5a63d4bdfbb81 I0704 22:23:24.539691 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=26f6ba0bea1807e743f061f92c38819791411699ed93bfb0e547d7c3f26f3076 I0704 22:23:24.539767 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=20e9f09a1fd11aa15b7bb75f09ab2aeefe49dfa298a42204e442388ba2b34257 I0704 22:23:24.539831 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=405a3686481e992aacacd3d4a7218099242264a6652573354f21acef30919c58 I0704 22:23:24.539909 1 recorder.go:70] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=291ba50266ba6fb225570c463c2bc88e070c45ebc809485fa18bc77957f91cb7 I0704 22:23:24.539961 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=27282bee0d9e7a043f89488319d3d87d700063425e7402fb3d8f9ca2cc04d7ed I0704 22:23:24.540006 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 42.844622ms to process 11 records I0704 22:23:24.540048 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0704 22:23:24.540095 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 31.307587ms to process 1 records I0704 22:23:24.540137 1 tasks_processing.go:74] worker 2 stopped. I0704 22:23:24.541011 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0704 22:23:24.541011 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0704 22:23:24.541113 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0704 22:23:24.543458 1 tasks_processing.go:74] worker 18 stopped. I0704 22:23:24.543471 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 46.329889ms to process 0 records I0704 22:23:24.545462 1 tasks_processing.go:74] worker 11 stopped. E0704 22:23:24.545473 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0704 22:23:24.545493 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0704 22:23:24.545502 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 39.654736ms to process 1 records I0704 22:23:24.546803 1 base_controller.go:82] Caches are synced for ConfigController I0704 22:23:24.546819 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0704 22:23:24.548864 1 tasks_processing.go:74] worker 23 stopped. I0704 22:23:24.548881 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 15.865869ms to process 0 records I0704 22:23:24.548897 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 16.45165ms to process 0 records I0704 22:23:24.548905 1 tasks_processing.go:74] worker 3 stopped. I0704 22:23:24.548928 1 tasks_processing.go:74] worker 19 stopped. I0704 22:23:24.549028 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=b6288b5e98b95f37df5f4e24f1384d47ca11f14242948c331a56b9ca76b26925 I0704 22:23:24.549046 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=18f22a295b86ea29ec1aeb57a28f0d56a1d52b109afb4127237f0845fb72be41 I0704 22:23:24.549052 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 28.468804ms to process 2 records I0704 22:23:24.549121 1 controller.go:203] Source scaController *sca.Controller is not ready I0704 22:23:24.549136 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0704 22:23:24.549141 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0704 22:23:24.549144 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0704 22:23:24.549147 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0704 22:23:24.549163 1 controller.go:458] The operator is still being initialized I0704 22:23:24.549170 1 controller.go:481] The operator is healthy I0704 22:23:24.551543 1 tasks_processing.go:74] worker 1 stopped. I0704 22:23:24.551678 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=dc5bdc84bcc6d7e1a7ff5573211fb918fdcf8355736e3cb1ca40c343de7a842c I0704 22:23:24.551740 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=ec8df744cf1f1b204deb43da83441ecb1906b567a2ea190b03412eae41044aa1 I0704 22:23:24.551794 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=02957d6a5361ef7ef7798d7dc6b90b368f8b1019dce62fb5344427e4e9b6d736 I0704 22:23:24.551806 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 19.194676ms to process 3 records I0704 22:23:24.554956 1 tasks_processing.go:74] worker 7 stopped. I0704 22:23:24.555019 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 47.276721ms to process 0 records I0704 22:23:24.557451 1 tasks_processing.go:74] worker 26 stopped. I0704 22:23:24.557947 1 recorder.go:70] Recording config/version with fingerprint=f934fad2b1f8cdc5d9a820ac2260b733490b46ff0f9d9bb3a5d65b4554208b5f I0704 22:23:24.557965 1 recorder.go:70] Recording config/id with fingerprint=6b0aa8c6a7b069483b367f3b439d0537fc8b524515416759493183ee3739a117 I0704 22:23:24.557975 1 gather.go:177] gatherer "clusterconfig" function "version" took 66.116338ms to process 2 records I0704 22:23:24.559285 1 tasks_processing.go:74] worker 10 stopped. I0704 22:23:24.559869 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=5e8d4860b298977f6e98d23f4b2c361c9f6935010b72125df948cce3a8dec770 I0704 22:23:24.560109 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=4a19b9a00bac7a736a88ca392fe691c6a11e9cba506fe8d838d8c4d7c3de4d2d I0704 22:23:24.560127 1 gather.go:177] gatherer "clusterconfig" function "crds" took 68.129022ms to process 2 records I0704 22:23:24.563114 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 22:23:24.563811 1 tasks_processing.go:74] worker 4 stopped. I0704 22:23:24.563890 1 recorder.go:70] Recording config/olm_operators with fingerprint=ebe327ba3bd7bb47f79dcef7207b7b9055eb612184f8cc484d075d64188e2710 I0704 22:23:24.563910 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 73.187326ms to process 1 records I0704 22:23:24.570863 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0704 22:23:24.572248 1 tasks_processing.go:74] worker 25 stopped. I0704 22:23:24.572268 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 52.369665ms to process 0 records I0704 22:23:24.580696 1 tasks_processing.go:74] worker 30 stopped. I0704 22:23:24.582674 1 recorder.go:70] Recording config/pod/openshift-console-operator/console-operator-85b84f584d-fv928 with fingerprint=3cb5695b3514125139c7a767127eac1bbc5efdf5383596ebf6a9730a6bf38f1e I0704 22:23:24.583043 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-ccsj6 with fingerprint=7765d5a7a76de440d91b23e5002a27c2ff91cb96ac973fa5c41dddf1b49c0a29 I0704 22:23:24.583168 1 recorder.go:70] Recording config/running_containers with fingerprint=3c5263cb9e761e5ccfa19ca0c49d47e02cccbac293f31c7d76a2cb39b869afd3 I0704 22:23:24.583186 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 73.129383ms to process 3 records I0704 22:23:24.585588 1 tasks_processing.go:74] worker 6 stopped. I0704 22:23:24.585600 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 67.537587ms to process 0 records I0704 22:23:24.586509 1 tasks_processing.go:74] worker 31 stopped. E0704 22:23:24.586521 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0704 22:23:24.586527 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0704 22:23:24.586530 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0704 22:23:24.586539 1 recorder.go:70] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=295a2bc7810c628251f1f312c209b5eb4da888326706c635ac0c12aec0969e17 I0704 22:23:24.586572 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0704 22:23:24.586580 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0704 22:23:24.586584 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=fe7abfa7f7aea852d8bca6b7df9b4d5de32045254ff2774f9bd30f0b6dcb7dc4 I0704 22:23:24.586593 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0704 22:23:24.586630 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0704 22:23:24.586638 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0704 22:23:24.586644 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 95.146323ms to process 7 records I0704 22:23:24.589299 1 tasks_processing.go:74] worker 21 stopped. E0704 22:23:24.589312 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0704 22:23:24.589319 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rbj8kks6i51bo7vb1cf526ol5phru77-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rbj8kks6i51bo7vb1cf526ol5phru77-primary-cert-bundle-secret" not found I0704 22:23:24.589370 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=4d679c7bf336922c8d4d383f6fd068183aaa980ecc1a68f46559b963c8f8bf39 I0704 22:23:24.589380 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 98.72534ms to process 1 records I0704 22:23:24.607334 1 base_controller.go:82] Caches are synced for LoggingSyncer I0704 22:23:24.607349 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0704 22:23:24.611235 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-56ffv pod in namespace openshift-dns (previous: false). I0704 22:23:24.626632 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress-operator" I0704 22:23:24.627250 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-f59dj pod in namespace openshift-dns (previous: false). I0704 22:23:24.628929 1 cluster_transfer.go:102] no available accepted cluster transfer I0704 22:23:24.628942 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0704 22:23:24.630466 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress" I0704 22:23:24.644273 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-f59dj pod in namespace openshift-dns (previous: false). I0704 22:23:24.659756 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-w6ngb pod in namespace openshift-dns (previous: false). I0704 22:23:24.673967 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-w6ngb pod in namespace openshift-dns (previous: false). I0704 22:23:24.686765 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-hglpr pod in namespace openshift-dns (previous: false). I0704 22:23:24.756866 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-szs2v pod in namespace openshift-dns (previous: false). I0704 22:23:24.783227 1 sca.go:266] etc-pki-entitlement secret successfully created I0704 22:23:24.783241 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0704 22:23:24.958167 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0704 22:23:24.958181 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-v5fq4 pod in namespace openshift-dns (previous: false). I0704 22:23:25.156473 1 tasks_processing.go:74] worker 12 stopped. I0704 22:23:25.156578 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=283d8ea38eb98498c6589d523a9cf6861faa6dd55588ec1a979c0af9eb70f5f9 I0704 22:23:25.156681 1 recorder.go:70] Recording events/openshift-dns with fingerprint=d62301ed97fb299374d2153f76c091f94f95190c98d3639cb43bd6d35c03f089 I0704 22:23:25.156692 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-56ffv/dns_current.log with fingerprint=00a357ef464d85bcffc09fcdde1c5719da4e900ee7eea62dc447fc20d4c101c3 I0704 22:23:25.156700 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-56ffv/kube-rbac-proxy_current.log with fingerprint=de8224507fe79062330e36530371f6750b4ea308557307a2e48092de238c20ab I0704 22:23:25.156706 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-f59dj/dns_current.log with fingerprint=a5b0f68d7f07459a70bdd296b8f4d00fd35c9a73307953c6eb0c0286835174c4 I0704 22:23:25.156713 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-f59dj/kube-rbac-proxy_current.log with fingerprint=5a15fc2ae7eb0577458b95775d6622c50faa825d3e9b86859b688ec85c0c0b77 I0704 22:23:25.156719 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-w6ngb/dns_current.log with fingerprint=5d690e534a6c62c23804b4c2501e8837101a00fa0f3a8d10ca1faba2d83da7bb I0704 22:23:25.156727 1 recorder.go:70] Recording config/pod/openshift-dns/logs/dns-default-w6ngb/kube-rbac-proxy_current.log with fingerprint=829e201df9cc41b63c178918e31bca2a7ba5b7f0f1840f016cf50622e596db24 I0704 22:23:25.156732 1 recorder.go:70] Recording config/pod/openshift-dns/logs/node-resolver-hglpr/dns-node-resolver_current.log with fingerprint=cad7143ac8fdc15dc8fd6d5274736716b7dcc082be2d9f452c1928f830255ab6 I0704 22:23:25.156737 1 recorder.go:70] Recording config/pod/openshift-dns/logs/node-resolver-v5fq4/dns-node-resolver_current.log with fingerprint=f9d62c3f78382fbdf345448b160e8dddae9ea1531cef45f73539b16fca7e2d8d I0704 22:23:25.156744 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 663.71863ms to process 10 records I0704 22:23:25.361560 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0704 22:23:25.550237 1 tasks_processing.go:74] worker 28 stopped. I0704 22:23:25.550268 1 recorder.go:70] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0704 22:23:25.550280 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.034633813s to process 1 records I0704 22:23:25.550425 1 tasks_processing.go:74] worker 22 stopped. I0704 22:23:25.550492 1 recorder.go:70] Recording config/alerts with fingerprint=fdfc59fc53a942c2a9f68c8c6a6a750453ea44c9c376b0c257b9ebfd76c80112 I0704 22:23:25.550507 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.043224382s to process 1 records I0704 22:23:25.561471 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0704 22:23:26.965923 1 tasks_processing.go:74] worker 20 stopped. I0704 22:23:26.966053 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=f8cf623a2547292136dcf1aaf04077039da42cbb18ff926c50a8dc5ef0203324 I0704 22:23:26.966106 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0704 22:23:26.966131 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=4bc162efd4cbe5ab312c2e46ba668de25db8cd24f6c5886c5245f7d3bffd299b I0704 22:23:26.966142 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0704 22:23:26.966165 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=2abfa91bca9d9f6eb27bed885ce21b937ff296e4291e66cac6d7b4909a5651e5 I0704 22:23:26.966183 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0704 22:23:26.966208 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=98b627443a32183e7bc36652f33658856b597cef555d610543c8e2bc8ff11d50 I0704 22:23:26.966234 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=14aa5a20a541842eb1d8d89dd1067bc99ace85191f58639049c24bf22e8918b2 I0704 22:23:26.966260 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=5925b3c8ce52900c976da222c065e5470f3f4027dc0c0eb260d4fabc703d4f2e I0704 22:23:26.966289 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=8cc95be79bb9152f1e9a54adf236620d215eba5857d2d5db1872397ea5c9453f I0704 22:23:26.966591 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=f02ffc89791d1b5fca9baee4794f41fefc6b3ea305c5e94fa9a1d2cea9755eac I0704 22:23:26.966614 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=0c6e1704308f984adb0b732ded9da0648413a0e68c9b9a63e4f7fc17c73c8325 I0704 22:23:26.966624 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0704 22:23:26.966642 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=ffa110184c7dea852218a06259bee2dfa43db4c662decf4a47f66b25c64948c1 I0704 22:23:26.966654 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0704 22:23:26.966671 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=818b042982e549a15dee70a99e05de1fbbaf179d537fd0ec4df53ba7f1a8e686 I0704 22:23:26.966681 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0704 22:23:26.966700 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=aeb293eba9fff69667a4bba770873a47aa9efa4b472178401801b55a026b5df4 I0704 22:23:26.966711 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0704 22:23:26.966759 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=2886a519f037d1d60a8d52b92ac2beced0e85d7b9f1c3d1ed0afe77dbb61b88f I0704 22:23:26.966860 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=e1e31c284f32376b2a6e2bf2f716077483757301cc037e254eded72109af333b I0704 22:23:26.966886 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0704 22:23:26.966908 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=1fedbde4acd537f661fa107752a0757c4939681bb9b0ac7c0a1767da4013614c I0704 22:23:26.966934 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=ab48d8e3b9100120f18cf2904cc460bea089acbda7b85808c3bf6bfbc4d81ca9 I0704 22:23:26.966944 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0704 22:23:26.966961 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=67407c022d692508609e0c4e6b2154fbe0eadfcffe7aab6224cf4fd14e9e4473 I0704 22:23:26.966970 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0704 22:23:26.966988 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=60a8fc007b02243be9581b4f4a236b93f57e2b936d2f4a1e51bcd09bcf99796f I0704 22:23:26.967007 1 recorder.go:70] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0704 22:23:26.967026 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=123c3c9962c5bd0c1c14c8f5bdcd0a6578880024a6463e97d17ec048139bf945 I0704 22:23:26.967046 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=3ce5c53ad96e8d8128d93b6960b861ced2f81847d49ade5d8c18b9350f39ed89 I0704 22:23:26.967063 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=a5f0cf94ecd01207c364bdd32271222ac8495ec26701b5b612c3d5d994be9872 I0704 22:23:26.967089 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=fa081109be7864f51935801c67264254a04c91e26ceeb30b23ee13f4bd6f8caa I0704 22:23:26.967100 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0704 22:23:26.967124 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=ebc0f41d7e702fdfe5f974c1666e2ebda559d9fe56f0fbaea1d39b51a53b9a06 I0704 22:23:26.967145 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0704 22:23:26.967155 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0704 22:23:26.967164 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.47618125s to process 37 records I0704 22:23:37.570879 1 tasks_processing.go:74] worker 15 stopped. I0704 22:23:37.570936 1 recorder.go:70] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0704 22:23:37.570953 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.035340706s to process 1 records I0704 22:23:38.300370 1 tasks_processing.go:74] worker 27 stopped. I0704 22:23:38.300704 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=dabcdda6991bac1666e8c7d62a42c20e4eacf733a346e4ee546090403d855ed3 I0704 22:23:38.300727 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.81245555s to process 1 records E0704 22:23:38.300788 1 periodic.go:250] "Unhandled Error" err="clusterconfig failed after 13.813s with: function \"pod_network_connectivity_checks\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"machines\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error" I0704 22:23:38.301899 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "machines" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error I0704 22:23:38.301914 1 periodic.go:212] Running workloads gatherer I0704 22:23:38.301931 1 tasks_processing.go:45] number of workers: 2 I0704 22:23:38.301944 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 22:23:38.301950 1 tasks_processing.go:71] worker 1 working on workload_info task. I0704 22:23:38.301953 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 22:23:38.302027 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0704 22:23:38.311714 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-131.ec2.internal... I0704 22:23:38.311729 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-90.ec2.internal... I0704 22:23:38.311729 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-120.ec2.internal... I0704 22:23:38.341031 1 tasks_processing.go:74] worker 0 stopped. I0704 22:23:38.341049 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 38.983932ms to process 0 records I0704 22:23:43.098270 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0704 22:23:43.126109 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0704 22:23:43.140214 1 gather_workloads_info.go:387] No image sha256:a7c71f3c9714cf1717c45e12ec817be6d6209f00989e6a31d634e527f0c8147d (15ms) I0704 22:23:43.153406 1 gather_workloads_info.go:387] No image sha256:bae7a33f8db8a3d4b3c4c05498aba85a0ce463f85322067c48e79663710e616e (13ms) I0704 22:23:43.166833 1 gather_workloads_info.go:387] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (13ms) I0704 22:23:43.179725 1 gather_workloads_info.go:387] No image sha256:9697bc2258bdfa9ae8c1866cc7eb0b3b46851998db827b121bdf77417a881eb3 (13ms) I0704 22:23:43.194454 1 gather_workloads_info.go:387] No image sha256:38dd36da63b1f01316a02daf6c18b821318c6ec313712147292d74abf955e0da (15ms) I0704 22:23:43.207218 1 gather_workloads_info.go:387] No image sha256:8ef4bdf07dda423fab73484dbc66d527ce8ba8cc2d4f99210bc1bc24ea08c0cb (13ms) I0704 22:23:43.220121 1 gather_workloads_info.go:387] No image sha256:799f5628c18e77a49c08645c03179046b5200c8cbcfd65737fb59668132f393e (13ms) I0704 22:23:43.232953 1 gather_workloads_info.go:387] No image sha256:87efe06afd1f04426fca7f86c0f74c4ee75c311ba199ddabd5b849b877bc59fa (13ms) I0704 22:23:43.245506 1 gather_workloads_info.go:387] No image sha256:6277dac65decba0ea094146bb99600f479eaf804696842e724de1f1b436f9a38 (13ms) I0704 22:23:43.257701 1 gather_workloads_info.go:387] No image sha256:1d471ad866b8564b209214d7dd3a7363c47e9535211d660b63b46abb278af7ab (12ms) I0704 22:23:43.270497 1 gather_workloads_info.go:387] No image sha256:e40bbab1e52f580ab471bd7ab19ae23a4bcbaf1893243a05238af6bede7ba9cc (13ms) I0704 22:23:43.338503 1 gather_workloads_info.go:387] No image sha256:752be69b2262be713df12c47f4bac8c2dafed272c401e9a89f8060f053d68054 (68ms) I0704 22:23:43.439284 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (101ms) I0704 22:23:43.538846 1 gather_workloads_info.go:387] No image sha256:5cba745f09f3bd6b8028c6773597cbb30bf3639a84f2b4b6493fe20642339a2c (100ms) I0704 22:23:43.638545 1 gather_workloads_info.go:387] No image sha256:944d9261ba7a143131fe8267c172defc5f37acc1cea3d4d373ec6fc5d8bfcc31 (100ms) I0704 22:23:43.740585 1 gather_workloads_info.go:387] No image sha256:0a92dd43975f972e3dc707fb37854e046813531553038147e9c90d54a8d9df73 (102ms) I0704 22:23:43.838966 1 gather_workloads_info.go:387] No image sha256:00f977c2275f5bf03d423911159a39ca44fabc1862ecb7ae492198ec8c46c6bb (98ms) I0704 22:23:43.938856 1 gather_workloads_info.go:387] No image sha256:14703f73bb1ca69ea03b726e340b3b68f8e294e1f22c80d1a16666ea3d4a88a3 (100ms) I0704 22:23:44.038693 1 gather_workloads_info.go:387] No image sha256:3d0634fe58641d5242649c44ddba70ed67fdd6d2dcb5c2261df5cee8b33de9fd (100ms) I0704 22:23:44.138586 1 gather_workloads_info.go:387] No image sha256:420c12f61c53e54eab2d99476759c0de339fca98ce0a3a782bc6545cc0e97a9c (100ms) I0704 22:23:44.239227 1 gather_workloads_info.go:387] No image sha256:46c2c396ca651699738c3f26b79b10af2922123dd27c2e32382d2a31a87e92c7 (101ms) I0704 22:23:44.341095 1 gather_workloads_info.go:387] No image sha256:7854174b4b98e9b10d9c8e9b43113df1217ab21db4b2395d9399af0c3d7911a3 (102ms) I0704 22:23:44.438707 1 gather_workloads_info.go:387] No image sha256:0a8a0473029ace3adbb66f490dbe560f0e7782a38cdf32c1f7dd3e092e1d191e (98ms) I0704 22:23:44.538997 1 gather_workloads_info.go:387] No image sha256:bcca5c17371caa2dce7e28da4c9fe30163c340fa172187f94db90ea1bd6ab80d (100ms) I0704 22:23:44.638579 1 gather_workloads_info.go:387] No image sha256:ccdda82b4adfa8ee4a84b3bad19693be6a42a9881a7b83788d835912129c49a4 (100ms) I0704 22:23:44.738896 1 gather_workloads_info.go:387] No image sha256:4f175ee49f51dc4379e9993fecd1657c7a9c4c3dc096772cb198d0212b9eea47 (100ms) I0704 22:23:44.838348 1 gather_workloads_info.go:387] No image sha256:270a7bac889ed64d75ce11ae26e8634a6209ff718b83fae2c888cfc758316aa3 (99ms) I0704 22:23:44.940887 1 gather_workloads_info.go:387] No image sha256:2fac754deaeade3456361eed52e344318ff16d04819384432759f0ea35530114 (103ms) I0704 22:23:45.038786 1 gather_workloads_info.go:387] No image sha256:18956d5fbc8d0369a26cb1e00eb5c3aff1973ae8cbd3983920a0cf2851f05358 (98ms) I0704 22:23:45.139200 1 gather_workloads_info.go:387] No image sha256:527ed405bf66234718321996dacf51f91c64c746de9f1509416ef6ce89c90a50 (100ms) I0704 22:23:45.238978 1 gather_workloads_info.go:387] No image sha256:a9e885d6f0456a2cc10f9e5da71fe9403f5e4c639b9b7ea15bd403b272ccb824 (100ms) I0704 22:23:45.338885 1 gather_workloads_info.go:387] No image sha256:76f820bd9bd138d29305d545d7d49bfe63e2923f1fb1ec2e8eac81a388359024 (100ms) I0704 22:23:45.438308 1 gather_workloads_info.go:387] No image sha256:8b1e32ce43eb1247b0558230b6e0baa85ac62f1a5e2a2089d40b4fea90538529 (99ms) I0704 22:23:45.539845 1 gather_workloads_info.go:398] Found image sha256:23c4fe84047a4e6cbe7f75f470e7d6fe0e61e7910dd17945b1b61bc4b72f3f2a (102ms) I0704 22:23:45.638823 1 gather_workloads_info.go:387] No image sha256:ce3d8211c0bde5788a4fd04c3eff31ae591437ceab2b3168dbbdfe408673c8b1 (99ms) I0704 22:23:45.738406 1 gather_workloads_info.go:387] No image sha256:1f2aa1e1026570162984f7fbb634635bcf319e408cafa8f9034ee842a2f193ab (100ms) I0704 22:23:45.839111 1 gather_workloads_info.go:387] No image sha256:68d8e92046969bcfcbca2d2b3dcf4dcad5f6edc5fcb44861eb22f8079a43ea4d (101ms) I0704 22:23:45.939178 1 gather_workloads_info.go:387] No image sha256:48404bd61d05dc738ccce4d22e36b30dbcdc6015b06b4afb604fd1baeee35bf2 (100ms) I0704 22:23:46.039421 1 gather_workloads_info.go:387] No image sha256:373614619b9420b110d1508c5f17e066ce69c4c226fd04d02fbb959d9ba41eb6 (100ms) I0704 22:23:46.139429 1 gather_workloads_info.go:387] No image sha256:2d077e73dd76873ee2a1583aa3b3e76a1b408737f0af2e22b2aa055604d89e81 (100ms) I0704 22:23:46.244196 1 gather_workloads_info.go:387] No image sha256:824a3b8f78e19aa21a9f6444aefe8e0b624886cca18fe828b13acbab55e6e868 (105ms) I0704 22:23:46.342032 1 gather_workloads_info.go:387] No image sha256:6d6ebff54b8adac74f4d1b12ac8aa0f16cd7b28370e0d6aa847d1c457e03a5b6 (98ms) I0704 22:23:46.442201 1 gather_workloads_info.go:387] No image sha256:929a4a9c96a08b57fe4c315deca55c30d1990c83e34b8b5692f7646d29f715c8 (100ms) I0704 22:23:46.539532 1 gather_workloads_info.go:387] No image sha256:7941b5d9a758b8667e99cd9236f7f96ec036af61df5cbc96e16077d36700d7c7 (97ms) I0704 22:23:46.639519 1 gather_workloads_info.go:387] No image sha256:6c3c7f77d7a3815bbb69cad5f6f1742c17d9597bc135ba3d116c9736d98fae33 (100ms) I0704 22:23:46.739631 1 gather_workloads_info.go:398] Found image sha256:59db91eadf1e1e5343777be43ff25fb712f300f9341be48ba05f45270944536f (100ms) I0704 22:23:46.838201 1 gather_workloads_info.go:387] No image sha256:039479f355d6903e1018abf8427fece3465c4348d4de2144ec38114b51bec27e (98ms) I0704 22:23:46.938191 1 gather_workloads_info.go:387] No image sha256:1cc2f3c83068c8c67d9070f67677e1bb6b4feff7ac5744cad08711893e56b55f (100ms) I0704 22:23:47.038312 1 gather_workloads_info.go:387] No image sha256:a3d34d8f38a540ce79d34a5248b1965346155497db57b72cb4758d659418059c (100ms) I0704 22:23:47.038345 1 tasks_processing.go:74] worker 1 stopped. I0704 22:23:47.038674 1 recorder.go:70] Recording config/workload_info with fingerprint=9b5f60db27c8fa80c16754e90cf9744044a415bc1ae3d35a049a15dae10f31d0 I0704 22:23:47.038692 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.736387951s to process 1 records I0704 22:23:47.038714 1 periodic.go:259] Periodic gather workloads completed in 8.736s I0704 22:23:47.038724 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0704 22:23:47.038728 1 periodic.go:212] Running conditional gatherer I0704 22:23:47.049267 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules I0704 22:23:47.202224 1 conditional_gatherer.go:336] updating alerts cache for conditional gatherer I0704 22:23:48.282119 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0704 22:23:48.293359 1 conditional_gatherer.go:390] cluster version is '4.19.9' I0704 22:23:48.293405 1 tasks_processing.go:45] number of workers: 3 I0704 22:23:48.293423 1 tasks_processing.go:69] worker 2 listening for tasks. I0704 22:23:48.293428 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0704 22:23:48.293485 1 tasks_processing.go:69] worker 1 listening for tasks. I0704 22:23:48.293496 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0704 22:23:48.293508 1 tasks_processing.go:69] worker 0 listening for tasks. I0704 22:23:48.293516 1 tasks_processing.go:74] worker 0 stopped. I0704 22:23:48.293532 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0704 22:23:48.293548 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.197µs to process 1 records I0704 22:23:48.293502 1 tasks_processing.go:71] worker 1 working on remote_configuration task. I0704 22:23:48.293589 1 tasks_processing.go:74] worker 1 stopped. I0704 22:23:48.293645 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0704 22:23:48.293659 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 2.828µs to process 1 records I0704 22:23:48.293793 1 tasks_processing.go:74] worker 2 stopped. I0704 22:23:48.293809 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 281.458µs to process 0 records I0704 22:23:48.293830 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=AsExpected message= I0704 22:23:48.293856 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0704 22:23:48.325126 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=f21b5d8e8be2054a39b45ac5f07da55e039bab0c9b1ae319d8487b56fc4e4592 I0704 22:23:48.325247 1 diskrecorder.go:70] Writing 120 records to /var/lib/insights-operator/insights-2026-07-04-222348.tar.gz I0704 22:23:48.332585 1 diskrecorder.go:51] Wrote 120 records to disk in 7ms I0704 22:23:48.332615 1 periodic.go:281] Gathering cluster info every 2h0m0s I0704 22:23:48.332637 1 periodic.go:282] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0704 22:24:54.488489 1 diskrecorder.go:223] Found files to send: insights-2026-07-04-222348.tar.gz I0704 22:24:54.488517 1 insightsuploader.go:150] Checking archives to upload periodically every 15m48.512338853s I0704 22:24:54.488525 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0704 22:24:54.507654 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0704 22:24:54.662486 1 requests.go:87] Successfully reported id=2026-07-04T22:24:54Z x-rh-insights-request-id=fdb547e1c35943a4afaf689512c40c7c, wrote=57771 I0704 22:24:54.662536 1 insightsuploader.go:187] Uploaded report successfully in 174.00241ms I0704 22:24:54.662556 1 controller.go:119] Initializing last reported time to 2026-07-04T22:24:54Z I0704 22:24:54.662578 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0704 22:24:54.662583 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0704 22:24:54.662589 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0704 22:24:54.670536 1 controller.go:481] The operator is healthy I0704 22:25:24.515164 1 controller.go:481] The operator is healthy I0704 22:25:58.774997 1 insightsreport.go:137] Pulling report from smart-proxy I0704 22:25:58.775022 1 insightsreport.go:149] Retrieving report I0704 22:25:58.786934 1 requests.go:110] Retrieving report for cluster: 8614a76f-d48e-43d2-a53f-b461e2fbb8d5 I0704 22:25:58.786946 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/8614a76f-d48e-43d2-a53f-b461e2fbb8d5/reports I0704 22:25:58.790034 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/8614a76f-d48e-43d2-a53f-b461e2fbb8d5/reports I0704 22:25:58.938592 1 insightsreport.go:184] Report retrieved I0704 22:25:58.988872 1 insightsreport.go:239] Report retrieved correctly I0704 22:27:24.517647 1 controller.go:481] The operator is healthy I0704 22:28:24.476696 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 22:28:24.492013 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:28:24.492032 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 22:28:24.503739 1 secretconfigobserver.go:119] support secret does not exist I0704 22:29:24.515645 1 controller.go:481] The operator is healthy I0704 22:31:24.516057 1 controller.go:481] The operator is healthy I0704 22:33:24.466739 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 22:33:24.473303 1 configmapobserver.go:84] configmaps "insights-config" not found I0704 22:33:24.504431 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 22:33:24.511251 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:33:24.511269 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 22:33:24.513835 1 controller.go:481] The operator is healthy I0704 22:33:24.517677 1 secretconfigobserver.go:119] support secret does not exist I0704 22:35:24.515981 1 controller.go:481] The operator is healthy I0704 22:37:24.515498 1 controller.go:481] The operator is healthy I0704 22:38:24.517861 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0704 22:38:24.525997 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0704 22:38:24.526015 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0704 22:38:24.532510 1 secretconfigobserver.go:119] support secret does not exist I0704 22:39:24.515683 1 controller.go:481] The operator is healthy I0704 22:40:48.175509 1 insightsuploader.go:146] Nothing to report since 2026-07-04T22:24:54Z I0704 22:41:24.515083 1 controller.go:481] The operator is healthy