I0521 19:48:46.450884 1 cmd.go:253] Using service-serving-cert provided certificates I0521 19:48:46.451590 1 observer_polling.go:159] Starting file observer I0521 19:48:46.821560 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0521 19:48:46.821771 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0521 19:48:46.822706 1 secure_serving.go:57] Forcing use of http/1.1 only W0521 19:48:46.822745 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0521 19:48:46.822753 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0521 19:48:46.822765 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0521 19:48:46.822769 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0521 19:48:46.822773 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0521 19:48:46.822782 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0521 19:48:46.824285 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0521 19:48:46.827384 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0521 19:48:46.827407 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0521 19:48:46.827399 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0521 19:48:46.827411 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0521 19:48:46.827432 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0521 19:48:46.827434 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0521 19:48:46.827778 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0521 19:48:46.827977 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0521 19:48:46.828034 1 secure_serving.go:213] Serving securely on [::]:8443 I0521 19:48:46.827997 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"55bfb96e-5e00-4c6b-bdd6-adba447624e8", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0521 19:48:46.828067 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0521 19:48:46.832488 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0521 19:48:46.832537 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0521 19:48:46.832636 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0521 19:48:46.837525 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 19:48:46.837541 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0521 19:48:46.841723 1 secretconfigobserver.go:119] support secret does not exist I0521 19:48:46.846001 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 19:48:46.849814 1 secretconfigobserver.go:119] support secret does not exist I0521 19:48:46.853606 1 recorder.go:161] Pruning old reports every 4h16m47s, max age is 288h0m0s I0521 19:48:46.857760 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0521 19:48:46.857769 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0521 19:48:46.857777 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0521 19:48:46.857780 1 insightsreport.go:296] Starting report retriever I0521 19:48:46.857789 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0521 19:48:46.857795 1 periodic.go:209] Running clusterconfig gatherer I0521 19:48:46.857836 1 tasks_processing.go:45] number of workers: 64 I0521 19:48:46.857862 1 tasks_processing.go:69] worker 3 listening for tasks. I0521 19:48:46.857871 1 tasks_processing.go:69] worker 2 listening for tasks. I0521 19:48:46.857874 1 tasks_processing.go:69] worker 1 listening for tasks. I0521 19:48:46.857878 1 tasks_processing.go:69] worker 14 listening for tasks. I0521 19:48:46.857884 1 tasks_processing.go:69] worker 25 listening for tasks. I0521 19:48:46.857886 1 tasks_processing.go:69] worker 4 listening for tasks. I0521 19:48:46.857885 1 tasks_processing.go:69] worker 0 listening for tasks. I0521 19:48:46.857892 1 tasks_processing.go:69] worker 15 listening for tasks. I0521 19:48:46.857894 1 tasks_processing.go:69] worker 9 listening for tasks. I0521 19:48:46.857893 1 tasks_processing.go:69] worker 5 listening for tasks. I0521 19:48:46.857897 1 tasks_processing.go:69] worker 10 listening for tasks. I0521 19:48:46.857900 1 tasks_processing.go:69] worker 16 listening for tasks. I0521 19:48:46.857905 1 tasks_processing.go:71] worker 9 working on openstack_controlplanes task. I0521 19:48:46.857907 1 tasks_processing.go:71] worker 10 working on nodes task. I0521 19:48:46.857941 1 tasks_processing.go:69] worker 36 listening for tasks. I0521 19:48:46.857947 1 tasks_processing.go:69] worker 21 listening for tasks. I0521 19:48:46.857957 1 tasks_processing.go:71] worker 36 working on container_images task. I0521 19:48:46.857952 1 tasks_processing.go:69] worker 24 listening for tasks. I0521 19:48:46.857960 1 tasks_processing.go:69] worker 23 listening for tasks. I0521 19:48:46.857967 1 tasks_processing.go:69] worker 18 listening for tasks. I0521 19:48:46.857967 1 tasks_processing.go:69] worker 11 listening for tasks. I0521 19:48:46.857969 1 tasks_processing.go:69] worker 28 listening for tasks. I0521 19:48:46.857975 1 tasks_processing.go:69] worker 26 listening for tasks. I0521 19:48:46.857975 1 tasks_processing.go:69] worker 22 listening for tasks. I0521 19:48:46.857978 1 tasks_processing.go:69] worker 19 listening for tasks. I0521 19:48:46.857977 1 tasks_processing.go:69] worker 31 listening for tasks. I0521 19:48:46.857984 1 tasks_processing.go:69] worker 20 listening for tasks. I0521 19:48:46.857989 1 tasks_processing.go:69] worker 29 listening for tasks. I0521 19:48:46.857991 1 tasks_processing.go:69] worker 34 listening for tasks. I0521 19:48:46.857999 1 tasks_processing.go:69] worker 6 listening for tasks. I0521 19:48:46.857996 1 tasks_processing.go:69] worker 35 listening for tasks. I0521 19:48:46.858000 1 tasks_processing.go:69] worker 8 listening for tasks. I0521 19:48:46.858007 1 tasks_processing.go:69] worker 32 listening for tasks. I0521 19:48:46.858009 1 tasks_processing.go:71] worker 14 working on cluster_apiserver task. I0521 19:48:46.857904 1 tasks_processing.go:71] worker 5 working on machine_healthchecks task. I0521 19:48:46.858009 1 tasks_processing.go:69] worker 33 listening for tasks. I0521 19:48:46.858009 1 tasks_processing.go:69] worker 30 listening for tasks. I0521 19:48:46.857946 1 tasks_processing.go:69] worker 17 listening for tasks. I0521 19:48:46.858025 1 tasks_processing.go:69] worker 56 listening for tasks. I0521 19:48:46.858026 1 tasks_processing.go:69] worker 62 listening for tasks. I0521 19:48:46.858030 1 tasks_processing.go:69] worker 40 listening for tasks. I0521 19:48:46.858031 1 tasks_processing.go:69] worker 57 listening for tasks. I0521 19:48:46.858035 1 tasks_processing.go:69] worker 50 listening for tasks. I0521 19:48:46.858036 1 tasks_processing.go:71] worker 3 working on config_maps task. I0521 19:48:46.857984 1 tasks_processing.go:69] worker 12 listening for tasks. I0521 19:48:46.858044 1 tasks_processing.go:69] worker 41 listening for tasks. I0521 19:48:46.858040 1 tasks_processing.go:69] worker 53 listening for tasks. I0521 19:48:46.858045 1 tasks_processing.go:69] worker 51 listening for tasks. I0521 19:48:46.858049 1 tasks_processing.go:69] worker 59 listening for tasks. I0521 19:48:46.858049 1 tasks_processing.go:71] worker 1 working on sap_config task. I0521 19:48:46.858057 1 tasks_processing.go:69] worker 45 listening for tasks. I0521 19:48:46.858040 1 tasks_processing.go:71] worker 2 working on qemu_kubevirt_launcher_logs task. I0521 19:48:46.858060 1 tasks_processing.go:69] worker 46 listening for tasks. I0521 19:48:46.858056 1 tasks_processing.go:69] worker 44 listening for tasks. I0521 19:48:46.858059 1 tasks_processing.go:71] worker 15 working on machine_configs task. I0521 19:48:46.858063 1 tasks_processing.go:69] worker 43 listening for tasks. I0521 19:48:46.857991 1 tasks_processing.go:69] worker 13 listening for tasks. I0521 19:48:46.857986 1 tasks_processing.go:69] worker 7 listening for tasks. I0521 19:48:46.857987 1 tasks_processing.go:69] worker 27 listening for tasks. I0521 19:48:46.858003 1 tasks_processing.go:69] worker 54 listening for tasks. I0521 19:48:46.858009 1 tasks_processing.go:71] worker 4 working on proxies task. I0521 19:48:46.857909 1 tasks_processing.go:71] worker 16 working on silenced_alerts task. I0521 19:48:46.858145 1 tasks_processing.go:71] worker 54 working on nodenetworkconfigurationpolicies task. I0521 19:48:46.858034 1 tasks_processing.go:69] worker 47 listening for tasks. I0521 19:48:46.858157 1 tasks_processing.go:71] worker 21 working on node_logs task. I0521 19:48:46.858163 1 tasks_processing.go:71] worker 47 working on certificate_signing_requests task. I0521 19:48:46.858016 1 tasks_processing.go:69] worker 58 listening for tasks. I0521 19:48:46.858018 1 tasks_processing.go:69] worker 38 listening for tasks. I0521 19:48:46.858019 1 tasks_processing.go:69] worker 61 listening for tasks. I0521 19:48:46.858014 1 tasks_processing.go:71] worker 25 working on cost_management_metrics_configs task. I0521 19:48:46.858199 1 tasks_processing.go:71] worker 19 working on machine_config_pools task. I0521 19:48:46.858231 1 tasks_processing.go:71] worker 33 working on oauths task. I0521 19:48:46.858238 1 tasks_processing.go:71] worker 24 working on nodenetworkstates task. I0521 19:48:46.858241 1 tasks_processing.go:71] worker 31 working on tsdb_status task. I0521 19:48:46.858243 1 tasks_processing.go:71] worker 61 working on mutating_webhook_configurations task. I0521 19:48:46.858242 1 tasks_processing.go:71] worker 8 working on machine_sets task. I0521 19:48:46.858023 1 tasks_processing.go:69] worker 55 listening for tasks. I0521 19:48:46.858253 1 tasks_processing.go:71] worker 22 working on image task. I0521 19:48:46.858259 1 tasks_processing.go:71] worker 55 working on sap_datahubs task. I0521 19:48:46.858009 1 tasks_processing.go:69] worker 37 listening for tasks. I0521 19:48:46.858025 1 tasks_processing.go:71] worker 0 working on dvo_metrics task. I0521 19:48:46.858270 1 tasks_processing.go:71] worker 23 working on install_plans task. I0521 19:48:46.858245 1 tasks_processing.go:71] worker 28 working on lokistack task. I0521 19:48:46.858033 1 tasks_processing.go:69] worker 60 listening for tasks. I0521 19:48:46.858045 1 tasks_processing.go:69] worker 48 listening for tasks. I0521 19:48:46.858054 1 tasks_processing.go:69] worker 42 listening for tasks. I0521 19:48:46.858051 1 tasks_processing.go:69] worker 49 listening for tasks. I0521 19:48:46.858148 1 tasks_processing.go:71] worker 32 working on openshift_machine_api_events task. I0521 19:48:46.858379 1 tasks_processing.go:71] worker 30 working on monitoring_persistent_volumes task. I0521 19:48:46.858392 1 tasks_processing.go:71] worker 29 working on validating_webhook_configurations task. I0521 19:48:46.858400 1 tasks_processing.go:71] worker 58 working on ingress_certificates task. I0521 19:48:46.858418 1 tasks_processing.go:71] worker 51 working on jaegers task. I0521 19:48:46.858025 1 tasks_processing.go:69] worker 39 listening for tasks. I0521 19:48:46.858233 1 tasks_processing.go:71] worker 34 working on pod_network_connectivity_checks task. I0521 19:48:46.858583 1 tasks_processing.go:71] worker 17 working on schedulers task. I0521 19:48:46.858009 1 tasks_processing.go:69] worker 63 listening for tasks. I0521 19:48:46.858719 1 tasks_processing.go:71] worker 56 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0521 19:48:46.858731 1 tasks_processing.go:71] worker 63 working on feature_gates task. I0521 19:48:46.858739 1 tasks_processing.go:71] worker 18 working on olm_operators task. I0521 19:48:46.858765 1 tasks_processing.go:71] worker 38 working on support_secret task. I0521 19:48:46.858233 1 tasks_processing.go:71] worker 6 working on image_pruners task. I0521 19:48:46.858238 1 tasks_processing.go:71] worker 35 working on metrics task. I0521 19:48:46.858039 1 tasks_processing.go:69] worker 52 listening for tasks. I0521 19:48:46.859013 1 tasks_processing.go:71] worker 52 working on version task. I0521 19:48:46.859033 1 tasks_processing.go:71] worker 59 working on operators task. I0521 19:48:46.858249 1 tasks_processing.go:71] worker 26 working on sap_pods task. I0521 19:48:46.859091 1 tasks_processing.go:71] worker 37 working on openstack_dataplanenodesets task. I0521 19:48:46.859096 1 tasks_processing.go:71] worker 46 working on storage_cluster task. I0521 19:48:46.859102 1 tasks_processing.go:71] worker 49 working on machines task. I0521 19:48:46.859091 1 tasks_processing.go:71] worker 45 working on authentication task. I0521 19:48:46.859184 1 tasks_processing.go:71] worker 62 working on pdbs task. I0521 19:48:46.859250 1 tasks_processing.go:71] worker 42 working on openstack_dataplanedeployments task. I0521 19:48:46.859319 1 tasks_processing.go:71] worker 40 working on ceph_cluster task. I0521 19:48:46.859554 1 tasks_processing.go:71] worker 20 working on openstack_version task. I0521 19:48:46.859602 1 tasks_processing.go:71] worker 12 working on machine_autoscalers task. I0521 19:48:46.859641 1 tasks_processing.go:71] worker 41 working on service_accounts task. I0521 19:48:46.859594 1 tasks_processing.go:71] worker 57 working on operators_pods_and_events task. I0521 19:48:46.859084 1 tasks_processing.go:71] worker 48 working on aggregated_monitoring_cr_names task. I0521 19:48:46.859598 1 tasks_processing.go:71] worker 50 working on networks task. I0521 19:48:46.859622 1 tasks_processing.go:71] worker 39 working on image_registries task. I0521 19:48:46.858231 1 tasks_processing.go:71] worker 11 working on storage_classes task. I0521 19:48:46.859631 1 tasks_processing.go:71] worker 44 working on crds task. I0521 19:48:46.859642 1 tasks_processing.go:71] worker 53 working on clusterroles task. I0521 19:48:46.859656 1 tasks_processing.go:71] worker 27 working on infrastructures task. I0521 19:48:46.859660 1 tasks_processing.go:71] worker 13 working on container_runtime_configs task. I0521 19:48:46.859664 1 tasks_processing.go:71] worker 7 working on active_alerts task. I0521 19:48:46.859630 1 tasks_processing.go:71] worker 60 working on ingress task. I0521 19:48:46.859674 1 tasks_processing.go:71] worker 43 working on openshift_logging task. I0521 19:48:46.861799 1 tasks_processing.go:71] worker 9 working on overlapping_namespace_uids task. I0521 19:48:46.861846 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 3.883375ms to process 0 records I0521 19:48:46.862445 1 tasks_processing.go:74] worker 5 stopped. E0521 19:48:46.862457 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0521 19:48:46.862466 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 4.417697ms to process 0 records E0521 19:48:46.864587 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.111.73:9091: connect: connection refused E0521 19:48:46.864584 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.111.73:9091: connect: connection refused I0521 19:48:46.864606 1 tasks_processing.go:74] worker 31 stopped. E0521 19:48:46.864614 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.111.73:9091: connect: connection refused I0521 19:48:46.864623 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 6.355284ms to process 0 records E0521 19:48:46.864631 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.111.73:9091: connect: connection refused I0521 19:48:46.864652 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 5.70787ms to process 0 records I0521 19:48:46.864663 1 tasks_processing.go:74] worker 35 stopped. I0521 19:48:46.875302 1 tasks_processing.go:74] worker 28 stopped. I0521 19:48:46.875313 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 17.015812ms to process 0 records I0521 19:48:46.875407 1 tasks_processing.go:74] worker 24 stopped. I0521 19:48:46.875420 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 17.15838ms to process 0 records I0521 19:48:46.875432 1 tasks_processing.go:74] worker 1 stopped. I0521 19:48:46.875436 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 17.371572ms to process 0 records I0521 19:48:46.875449 1 tasks_processing.go:74] worker 34 stopped. E0521 19:48:46.875456 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0521 19:48:46.875462 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 16.967606ms to process 0 records I0521 19:48:46.875467 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 17.022748ms to process 0 records I0521 19:48:46.875471 1 tasks_processing.go:74] worker 51 stopped. I0521 19:48:46.875707 1 tasks_processing.go:74] worker 32 stopped. I0521 19:48:46.875723 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 17.351875ms to process 0 records I0521 19:48:46.875756 1 tasks_processing.go:74] worker 4 stopped. I0521 19:48:46.875908 1 recorder.go:75] Recording config/proxy with fingerprint=7ac1a5127f59a3c37b9e687bfbdb880f79db212ff003d5d917f4003f4705f682 I0521 19:48:46.875927 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 17.658576ms to process 1 records I0521 19:48:46.876006 1 tasks_processing.go:74] worker 14 stopped. I0521 19:48:46.876307 1 recorder.go:75] Recording config/apiserver with fingerprint=17f7c987c404dd9f55167c85433a111a98ceb4429f424546692d7be14f87d78b I0521 19:48:46.876320 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 17.784308ms to process 1 records I0521 19:48:46.876331 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 17.74488ms to process 0 records I0521 19:48:46.876337 1 tasks_processing.go:74] worker 30 stopped. I0521 19:48:46.876453 1 tasks_processing.go:74] worker 33 stopped. I0521 19:48:46.876886 1 recorder.go:75] Recording config/oauth with fingerprint=2ac91304311a84b9d19ef0f6f140926bfec0e995e07769aa0bc028e541d789d4 I0521 19:48:46.876905 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 18.206374ms to process 1 records I0521 19:48:46.876944 1 tasks_processing.go:74] worker 55 stopped. I0521 19:48:46.876961 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 18.675368ms to process 0 records I0521 19:48:46.876970 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 17.860738ms to process 0 records I0521 19:48:46.876979 1 tasks_processing.go:74] worker 37 stopped. I0521 19:48:46.877086 1 tasks_processing.go:74] worker 54 stopped. I0521 19:48:46.877112 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 18.929735ms to process 0 records E0521 19:48:46.877120 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0521 19:48:46.877127 1 gather.go:177] gatherer "clusterconfig" function "machines" took 17.976905ms to process 0 records I0521 19:48:46.877133 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 18.857567ms to process 0 records I0521 19:48:46.877146 1 tasks_processing.go:74] worker 49 stopped. I0521 19:48:46.877157 1 tasks_processing.go:74] worker 8 stopped. I0521 19:48:46.877200 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0521 19:48:46.877219 1 controller.go:326] The initial operator extension status is healthy I0521 19:48:46.877230 1 tasks_processing.go:74] worker 10 stopped. I0521 19:48:46.877230 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0521 19:48:46.877238 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0521 19:48:46.877243 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0521 19:48:46.877261 1 controller.go:489] The operator is still being initialized I0521 19:48:46.877272 1 controller.go:512] The operator is healthy I0521 19:48:46.877340 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0521 19:48:46.877350 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0521 19:48:46.877684 1 recorder.go:75] Recording config/node/ip-10-0-0-111.ec2.internal with fingerprint=84fd4036cf2a9458949b8d07d081d390fed87638f9a2826c6a8ee8ea62b9a104 I0521 19:48:46.877892 1 gather_logs.go:145] no pods in namespace were found I0521 19:48:46.877954 1 recorder.go:75] Recording config/node/ip-10-0-1-90.ec2.internal with fingerprint=71ac593e370910304d8b977031b1cb3efacfff25ac1851eb537952e23d1606b4 I0521 19:48:46.878097 1 recorder.go:75] Recording config/node/ip-10-0-2-228.ec2.internal with fingerprint=87c6fb10be5abfd49b1eb075c1b5b0bd17bb7bfedd6be0a3aece2a146c6fe4d5 I0521 19:48:46.878113 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 19.222478ms to process 3 records W0521 19:48:46.878130 1 operator.go:288] started I0521 19:48:46.878154 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0521 19:48:46.878243 1 tasks_processing.go:74] worker 63 stopped. I0521 19:48:46.878266 1 recorder.go:75] Recording config/featuregate with fingerprint=38f3c8bd5326e38da8299560c24601d15b2c2de27dc103052ff576e5c86f61b6 I0521 19:48:46.878277 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 18.514826ms to process 1 records I0521 19:48:46.878367 1 tasks_processing.go:74] worker 22 stopped. I0521 19:48:46.878451 1 recorder.go:75] Recording config/image with fingerprint=e42c49887b704bb9776944ae59cae2678a24ffd40ab5e73ff420ec13de39c2cf I0521 19:48:46.878462 1 gather.go:177] gatherer "clusterconfig" function "image" took 19.028727ms to process 1 records I0521 19:48:46.878469 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 17.676369ms to process 0 records I0521 19:48:46.878472 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 18.237724ms to process 0 records I0521 19:48:46.878542 1 tasks_processing.go:74] worker 12 stopped. I0521 19:48:46.878551 1 tasks_processing.go:74] worker 46 stopped. I0521 19:48:46.878548 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=0c48d723c06cbaca12c1f72d417475e53663fe16dfdec2d2d2251fb77a3699fa I0521 19:48:46.878553 1 tasks_processing.go:74] worker 17 stopped. I0521 19:48:46.878576 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 18.761733ms to process 1 records E0521 19:48:46.878592 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0521 19:48:46.878599 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 18.615832ms to process 0 records I0521 19:48:46.878604 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 18.147899ms to process 0 records I0521 19:48:46.878609 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 18.089553ms to process 0 records I0521 19:48:46.878618 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 19.295534ms to process 0 records I0521 19:48:46.878623 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 19.834035ms to process 0 records I0521 19:48:46.878627 1 tasks_processing.go:74] worker 42 stopped. I0521 19:48:46.878630 1 tasks_processing.go:74] worker 2 stopped. I0521 19:48:46.878636 1 tasks_processing.go:74] worker 40 stopped. I0521 19:48:46.878636 1 tasks_processing.go:74] worker 38 stopped. I0521 19:48:46.878647 1 tasks_processing.go:74] worker 21 stopped. I0521 19:48:46.878803 1 tasks_processing.go:74] worker 19 stopped. I0521 19:48:46.878819 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 20.588274ms to process 0 records I0521 19:48:46.878883 1 tasks_processing.go:74] worker 62 stopped. I0521 19:48:46.878996 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=91307ee06640e3c610fadf1191d9a9c6271a94327bb46f9a3e22edf40c7d000c I0521 19:48:46.879026 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=03fa8103c311804454f82963155bb18438338c3b08c21b50da4f6fd83441fa82 I0521 19:48:46.879044 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=96f6cc9836662b2cd463b4cdcc211226c9aa76dc74c4cddf9a3159cbf5b108a2 I0521 19:48:46.879061 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=cd6d79618525a9dd40c7b6cc9314b4eda77c02f722ddc22abbe69648b661999b I0521 19:48:46.879086 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=d871b40c73bd9af0175be2b88803d3e4304864126503ec1b294b122970b1c8c1 I0521 19:48:46.879102 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=8b2d1e23add865212eb0562311da7bbc4eb2a2b809e6b3daa47273088797cd4e I0521 19:48:46.879125 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=577e4dc7940503a3a88ff69e5629e7deed66a28e916996c46ccf0c67ccbbf64e I0521 19:48:46.879141 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=ffaf79d889aef4ecb983a10a4bd11a26a605abd31c6f1d790399eb1931d79d6b I0521 19:48:46.879164 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=3fbabc58cfb0fbd8c7a4aac5d4c8bbde69cd9a2e5323a4636e11601b1c15200c I0521 19:48:46.879180 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=8ba5cce22011b9971fae3f61ca3a75052b4c95fc76775b7ba3e93a10268fef3b I0521 19:48:46.879195 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=784a611ec969b70b672e245e72173c86b859ef920dcd75d960462a7be24ed81f I0521 19:48:46.879203 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 19.68086ms to process 11 records I0521 19:48:46.879296 1 tasks_processing.go:74] worker 45 stopped. I0521 19:48:46.879355 1 recorder.go:75] Recording config/authentication with fingerprint=66f0ff68ca5f22dc297e39624e61e05b74dd257e9e63e71287670bd66810384a I0521 19:48:46.879364 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 19.928701ms to process 1 records I0521 19:48:46.880628 1 tasks_processing.go:74] worker 20 stopped. I0521 19:48:46.880640 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 21.014889ms to process 0 records I0521 19:48:46.883962 1 tasks_processing.go:74] worker 25 stopped. I0521 19:48:46.883980 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 25.752709ms to process 0 records I0521 19:48:46.884866 1 tasks_processing.go:74] worker 0 stopped. I0521 19:48:46.884879 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0521 19:48:46.884887 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 26.587242ms to process 1 records I0521 19:48:46.885380 1 tasks_processing.go:74] worker 43 stopped. I0521 19:48:46.885397 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 25.166169ms to process 0 records I0521 19:48:46.888324 1 tasks_processing.go:74] worker 26 stopped. I0521 19:48:46.888339 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 29.232449ms to process 0 records I0521 19:48:46.890438 1 tasks_processing.go:74] worker 6 stopped. I0521 19:48:46.890655 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=dfd7c7d79d7b124b8920356b9d7dbb11eb165054f3ad78123bc461b64dd24422 I0521 19:48:46.890667 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 31.56018ms to process 1 records I0521 19:48:46.893698 1 tasks_processing.go:74] worker 39 stopped. I0521 19:48:46.894046 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=0f3a0cb80027a019a32599643357e769d7784cc7450febbc4256d76603628983 I0521 19:48:46.894060 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 33.730588ms to process 1 records I0521 19:48:46.900976 1 tasks_processing.go:74] worker 52 stopped. I0521 19:48:46.901631 1 recorder.go:75] Recording config/version with fingerprint=bde613a364e02758295d3d7cfb362678136e3d7a1e93753d6427785036678c6f I0521 19:48:46.901661 1 recorder.go:75] Recording config/id with fingerprint=9fbd57e92925a2484ac9f731b69e6af450287862b9d905006ebbbcae663513bc I0521 19:48:46.901670 1 gather.go:177] gatherer "clusterconfig" function "version" took 41.951376ms to process 2 records I0521 19:48:46.901814 1 tasks_processing.go:74] worker 18 stopped. I0521 19:48:46.901963 1 recorder.go:75] Recording config/olm_operators with fingerprint=c50a70981584e2393c0de6b2804caab274a70372fb5b65142a4f82ef33806fec I0521 19:48:46.902022 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 43.062605ms to process 1 records I0521 19:48:46.903361 1 tasks_processing.go:74] worker 13 stopped. I0521 19:48:46.903383 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 43.223503ms to process 0 records I0521 19:48:46.903704 1 tasks_processing.go:74] worker 50 stopped. I0521 19:48:46.903820 1 recorder.go:75] Recording config/network with fingerprint=2e39521355367c65c12507352df888a7267cdd966825cdb06567607b4fc8ebd4 I0521 19:48:46.903832 1 gather.go:177] gatherer "clusterconfig" function "networks" took 43.781752ms to process 1 records I0521 19:48:46.904319 1 tasks_processing.go:74] worker 11 stopped. I0521 19:48:46.904396 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=b366deb88135b88e6181d66fd3564f623d5f09c348347e1f111768e8436a9980 I0521 19:48:46.904418 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=73200448571f935d37ac0d9df57467c6a398fa5fd4cfd5f88191a278731434b2 I0521 19:48:46.904426 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 44.34139ms to process 2 records I0521 19:48:46.904573 1 tasks_processing.go:74] worker 27 stopped. I0521 19:48:46.904584 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0521 19:48:46.904594 1 controller.go:212] Source scaController *sca.Controller is not ready I0521 19:48:46.904599 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0521 19:48:46.904604 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0521 19:48:46.904608 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0521 19:48:46.904630 1 controller.go:489] The operator is still being initialized I0521 19:48:46.904642 1 controller.go:512] The operator is healthy I0521 19:48:46.905085 1 recorder.go:75] Recording config/infrastructure with fingerprint=dd4fb61f7e8e2ab65397f71635a8dd68d5cb7c6a79075f5f21aa519b5a401bc2 I0521 19:48:46.905110 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 44.470413ms to process 1 records I0521 19:48:46.905193 1 tasks_processing.go:74] worker 60 stopped. I0521 19:48:46.905212 1 recorder.go:75] Recording config/ingress with fingerprint=9aad52d02da0b6ba3461d831e613a0a0eb411815d1c52e743264506bbc92b0fd I0521 19:48:46.905220 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 44.440809ms to process 1 records I0521 19:48:46.905293 1 tasks_processing.go:74] worker 61 stopped. I0521 19:48:46.905351 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=067521404fa3c237ecf019f48ff9081e88ff7f23faba36dc9a7d761c1cc62a89 I0521 19:48:46.905379 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=6db1271621f2be3aec07975ef4a890688574766e43a77adb18836195aa118b23 I0521 19:48:46.905403 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=8b315d999b7f70d8478baeef9d3951cbca1e782799d650627d97916467f4a1fc I0521 19:48:46.905410 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 46.450896ms to process 3 records I0521 19:48:46.907745 1 tasks_processing.go:74] worker 9 stopped. I0521 19:48:46.907771 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0521 19:48:46.907780 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 45.880975ms to process 1 records I0521 19:48:46.908026 1 tasks_processing.go:74] worker 15 stopped. I0521 19:48:46.908055 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0521 19:48:46.908067 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 49.949398ms to process 1 records I0521 19:48:46.908336 1 tasks_processing.go:74] worker 44 stopped. I0521 19:48:46.908893 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=a1b83aa2636fa3ca83c34dea6471ca8ff9a9ad53f39d5905ab604f7f3fe98a8d I0521 19:48:46.909637 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=e4656196c7a49987c5a889a2064594acdb9207b997129c390d7ad53f04cae58e I0521 19:48:46.909666 1 gather.go:177] gatherer "clusterconfig" function "crds" took 48.340464ms to process 2 records I0521 19:48:46.909857 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=98092239d255b49a16e31d208e1ac3b8a86ef4e3a65825b56ae483a212ad8785 I0521 19:48:46.910412 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=9a0a207ade1ecdf8bcb63a68eabd2720accecb5a467827840a434f8753283e0d I0521 19:48:46.910572 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=24eadca925ddc49aa9a3a9faf01d94536a102298c41793c5bef3ba760a5c9815 I0521 19:48:46.910604 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=fd13b8cab7d4854a64d4c8d610d98ad783e733e535e89e393919d9cf0b627e25 I0521 19:48:46.910631 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=ae9a0dc4d43b12747b4903d417c2a8d42c9c2aca1471ab6bc6215efbb7db948f I0521 19:48:46.910658 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=c6530731f60a7e1574cc22a97685205f216d779e60cc317705432c43ade78258 I0521 19:48:46.910706 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=7fc1899a56b9bbf8a3aeaa1177d7e2c3efa0a08ca79481844828b6488472df19 I0521 19:48:46.910775 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=c7ea825c93d05399f08f021b80a23f8f2bde2e43e1c41040de4fb7a66ca77a83 I0521 19:48:46.910818 1 tasks_processing.go:74] worker 29 stopped. I0521 19:48:46.911538 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=f00300a6c958cd63ef7e25bb0aa5b08675f379f15409de5695e679f5019a019b I0521 19:48:46.911623 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=69572d389fb54226e44e30756595fa6d14b6b582f4401617422263e1464f74ff I0521 19:48:46.911672 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=11f56632408cf2fd6d769b5e5f94312a992b4fbf825aaaa5989d1d398f4b19a6 I0521 19:48:46.911708 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=c69da1a524f2b6b230eebdc11ad37a602dbde69c001dc107be26bad3e9eaaf59 I0521 19:48:46.911739 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=755307447aeabdd789c78ef30ecdd803fcca017a750800290b7f597892eaaeb6 I0521 19:48:46.911752 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 49.999934ms to process 13 records I0521 19:48:46.911772 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 51.125578ms to process 0 records I0521 19:48:46.911786 1 tasks_processing.go:74] worker 47 stopped. I0521 19:48:46.913328 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0521 19:48:46.914337 1 tasks_processing.go:74] worker 53 stopped. I0521 19:48:46.914509 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=c6aab96d2c5b0075ab4aa1a95bc86735f104e4840028b406836828899ee467b1 I0521 19:48:46.914590 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=61864eca7282afe23141a8a24dcbdcbfae288e1190c0a3b08637147f59684c99 I0521 19:48:46.914600 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 54.28643ms to process 2 records I0521 19:48:46.919238 1 tasks_processing.go:74] worker 36 stopped. I0521 19:48:46.922687 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-kxttn with fingerprint=6a12ddebd6f859df91c48a4e1fccd13078df483238bc93a1a4f72447062186d4 I0521 19:48:46.922783 1 recorder.go:75] Recording config/running_containers with fingerprint=612cd1fc2354d62a1e1746cbbe5dc43559b0c7ce8fc00b8dee3d7995b285b881 I0521 19:48:46.922793 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 61.271642ms to process 2 records I0521 19:48:46.922807 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 62.564329ms to process 0 records I0521 19:48:46.922814 1 tasks_processing.go:74] worker 48 stopped. I0521 19:48:46.922851 1 tasks_processing.go:74] worker 57 stopped. I0521 19:48:46.922867 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 63.01754ms to process 0 records I0521 19:48:46.928423 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0521 19:48:46.928434 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0521 19:48:46.928446 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0521 19:48:46.928969 1 tasks_processing.go:74] worker 58 stopped. E0521 19:48:46.928983 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0521 19:48:46.928992 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2qegov8hghmr96o71hq8vjc3pqf2l10s-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2qegov8hghmr96o71hq8vjc3pqf2l10s-primary-cert-bundle-secret" not found I0521 19:48:46.929038 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=8bd2d9f6ec3004e7ecd14b565150d726249ba0abf149900eb95081e024cb9d70 I0521 19:48:46.929064 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 70.557703ms to process 1 records I0521 19:48:46.932759 1 base_controller.go:82] Caches are synced for ConfigController I0521 19:48:46.932771 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0521 19:48:46.933348 1 tasks_processing.go:74] worker 56 stopped. I0521 19:48:46.933359 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 74.619947ms to process 0 records I0521 19:48:46.934033 1 tasks_processing.go:74] worker 3 stopped. E0521 19:48:46.934048 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0521 19:48:46.934054 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0521 19:48:46.934058 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0521 19:48:46.934067 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0521 19:48:46.934098 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0521 19:48:46.934108 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0521 19:48:46.934112 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0521 19:48:46.934122 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0521 19:48:46.934159 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0521 19:48:46.934166 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0521 19:48:46.934171 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 75.978571ms to process 7 records I0521 19:48:46.938115 1 configmapobserver.go:84] configmaps "insights-config" not found I0521 19:48:46.977201 1 cluster_transfer.go:102] no available accepted cluster transfer I0521 19:48:46.977217 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0521 19:48:46.978274 1 base_controller.go:82] Caches are synced for LoggingSyncer I0521 19:48:46.978287 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0521 19:48:47.076904 1 sca.go:266] etc-pki-entitlement secret successfully created I0521 19:48:47.076918 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0521 19:48:47.924143 1 tasks_processing.go:74] worker 7 stopped. I0521 19:48:47.924186 1 recorder.go:75] Recording config/alerts with fingerprint=c9ed24454682886a35bef8c008ca3b03e55cf6f0c9a738fcfb54390911970f29 I0521 19:48:47.924199 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.063964692s to process 1 records I0521 19:48:47.924279 1 tasks_processing.go:74] worker 16 stopped. I0521 19:48:47.924289 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0521 19:48:47.924294 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.066137104s to process 1 records I0521 19:48:49.331574 1 tasks_processing.go:74] worker 59 stopped. I0521 19:48:49.331684 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=39905a277d0674474703822355f1387a5b28d659615dec8fd3adc2a0e45a315c I0521 19:48:49.331730 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0521 19:48:49.331754 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=7ff124e57800fa7aadc7e01983565748c2c2bb6928761253cf0079c259355dea I0521 19:48:49.331764 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0521 19:48:49.331786 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=0fbe220bbd246b69111e916c64bfad81b4a0a3a3bf21b709a9b46263750fd4b0 I0521 19:48:49.331801 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0521 19:48:49.331821 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=eab3244a571207eede4742c0f82e742328a88087fae7140045f867982dae235a I0521 19:48:49.331849 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=80176bcf7da31ae211023848ac6b1b0e71b2381631d3e03b451596487a2927f8 I0521 19:48:49.331872 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=83843fa29592a8f8179d1eb717ba0086a7a797bd9c411ed84956c64b475fa2ba I0521 19:48:49.331905 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=bfd96c9ee4aec40984f138e0dae042485a19c956aca38bf444cafec089ec1074 I0521 19:48:49.332160 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=35ec6f5f64f7938d8f541ae5ac865db72f60ab0c5bfffdd4ff330d4ea6238c94 I0521 19:48:49.332180 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=f4096380dd0d53cf0be6615dcc31b8f99bfcf10800c7a336bdc958e00f160b9b I0521 19:48:49.332189 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0521 19:48:49.332205 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=0ff0171ffe1a9c271e53278420d4a901a73994713e84f57b76bd080d5070d8c0 I0521 19:48:49.332216 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0521 19:48:49.332230 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=1058a862174775fb6b3ef1556b6622554143b886ad16e83b662b8b65f7c55f93 I0521 19:48:49.332238 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0521 19:48:49.332257 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=0a684a8d7ba209ccc01991965d59f4993f1572648e0be58152d2e2746aa302c1 I0521 19:48:49.332266 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0521 19:48:49.332285 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=dde98440c5e758bcf55ecfa8249d3169fd165e936a13e644652582fa79ab228b I0521 19:48:49.332400 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=9a2e4c8b81a59d7bd7194de030dfb10be773d2a3c5e6cbf8b7f62922d2a39f91 I0521 19:48:49.332410 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0521 19:48:49.332417 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0521 19:48:49.332440 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0521 19:48:49.332461 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=e13774473eb92af0eb1ab6112631bf92e37ba577618f87890e2576dcd4081bfe I0521 19:48:49.332486 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=b6bf8a89824924042f58f4df992f6d0604cc44bd51dad85f1dd51fb6d024febe I0521 19:48:49.332511 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0521 19:48:49.332528 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=20da2405d760a662e0c3054b3d220bae5fd0eb730320179a0feac7bbd5c4c42f I0521 19:48:49.332538 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0521 19:48:49.332555 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=6e2a94420d126f1487fe3903b456c3e9ea6369e81ed2312aacac54bddc40e1fc I0521 19:48:49.332565 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0521 19:48:49.332578 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=88333156db138f7578994574cea2a34dcd2114b00a44528c171bc306597217c8 I0521 19:48:49.332593 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=c70060229ea9f30038f6126b2d29d9fd010909518ab9e5469af1ccea814180e9 I0521 19:48:49.332607 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=4fd37e2db20890d63c7b4c341a91a55635522d37519891df7352639de5f492d9 I0521 19:48:49.332630 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=103e44af8a67c9fee9dedef6e6077b844f88972a3d26b4a3b3673b7f863a7ec8 I0521 19:48:49.332640 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0521 19:48:49.332667 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=3c5a919cdafb830a336f3d203de3c8fda244b0aff184b8ac05cd713e50f63017 I0521 19:48:49.332683 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0521 19:48:49.332692 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0521 19:48:49.332699 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.472524216s to process 39 records I0521 19:48:59.885533 1 tasks_processing.go:74] worker 23 stopped. I0521 19:48:59.885578 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0521 19:48:59.885590 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.027245732s to process 1 records I0521 19:49:00.668076 1 tasks_processing.go:74] worker 41 stopped. I0521 19:49:00.668359 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=ae29e7659fb4cd0f04bcc181913d3f8b69c20a612f3ed6bf2f2479314a8cc58f I0521 19:49:00.668377 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.808365103s to process 1 records E0521 19:49:00.668434 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.81s with: function \"machine_healthchecks\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error" I0521 19:49:00.669520 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error I0521 19:49:00.669534 1 periodic.go:209] Running workloads gatherer I0521 19:49:00.669548 1 tasks_processing.go:45] number of workers: 2 I0521 19:49:00.669554 1 tasks_processing.go:69] worker 1 listening for tasks. I0521 19:49:00.669558 1 tasks_processing.go:71] worker 1 working on workload_info task. I0521 19:49:00.669562 1 tasks_processing.go:69] worker 0 listening for tasks. I0521 19:49:00.669578 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0521 19:49:00.675823 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-111.ec2.internal... I0521 19:49:00.675830 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-90.ec2.internal... I0521 19:49:00.675842 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-228.ec2.internal... I0521 19:49:00.694108 1 tasks_processing.go:74] worker 0 stopped. I0521 19:49:00.694127 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 24.510368ms to process 0 records I0521 19:49:05.319916 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0521 19:49:05.344119 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0521 19:49:05.352472 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0521 19:49:05.360886 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (8ms) I0521 19:49:05.368848 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0521 19:49:05.376935 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (8ms) I0521 19:49:05.384903 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (8ms) I0521 19:49:05.392758 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (8ms) I0521 19:49:05.400454 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (8ms) I0521 19:49:05.407839 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (7ms) I0521 19:49:05.415701 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (8ms) I0521 19:49:05.424988 1 gather_workloads_info.go:387] No image sha256:f68c0da8268f23078f5f760a91a3c8212f2195acf18c231f6e034207a67adeec (9ms) I0521 19:49:05.451525 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (27ms) I0521 19:49:05.552960 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0521 19:49:05.652147 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (99ms) I0521 19:49:05.752832 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (101ms) I0521 19:49:05.857081 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (104ms) I0521 19:49:05.952662 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (96ms) I0521 19:49:06.052977 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (100ms) I0521 19:49:06.152628 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0521 19:49:06.253048 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0521 19:49:06.353350 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0521 19:49:06.452214 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (99ms) I0521 19:49:06.553661 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0521 19:49:06.653887 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0521 19:49:06.752991 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (99ms) I0521 19:49:06.853075 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0521 19:49:06.952455 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (99ms) I0521 19:49:07.052170 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0521 19:49:07.152358 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0521 19:49:07.252603 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0521 19:49:07.353257 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0521 19:49:07.452508 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0521 19:49:07.555374 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (103ms) I0521 19:49:07.652435 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (97ms) I0521 19:49:07.753253 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (101ms) I0521 19:49:07.856159 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (103ms) I0521 19:49:07.953538 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (97ms) I0521 19:49:08.053211 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0521 19:49:08.153306 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0521 19:49:08.252169 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (99ms) I0521 19:49:08.352391 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0521 19:49:08.452624 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0521 19:49:08.552793 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0521 19:49:08.651604 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (99ms) I0521 19:49:08.753543 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (102ms) I0521 19:49:08.866279 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (113ms) I0521 19:49:08.959989 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (94ms) I0521 19:49:09.054942 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (95ms) I0521 19:49:09.153202 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (98ms) I0521 19:49:09.252647 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (99ms) I0521 19:49:09.352748 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0521 19:49:09.352780 1 tasks_processing.go:74] worker 1 stopped. I0521 19:49:09.353073 1 recorder.go:75] Recording config/workload_info with fingerprint=6d308482ea5ce802a809da2b8fdf7821431cf2c6087b1f40ba65b4fc8fba37c5 I0521 19:49:09.353088 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.683215752s to process 1 records I0521 19:49:09.353103 1 periodic.go:256] Periodic gather workloads completed in 8.683s I0521 19:49:09.353110 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0521 19:49:09.353122 1 periodic.go:209] Running conditional gatherer I0521 19:49:09.361334 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0521 19:49:09.457595 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0521 19:49:09.459891 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.111.73:9091: connect: connection refused I0521 19:49:09.459969 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0521 19:49:09.469253 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0521 19:49:09.469267 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469272 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469275 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469279 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469282 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469285 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469288 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469291 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469293 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0521 19:49:09.469298 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0521 19:49:09.469312 1 tasks_processing.go:45] number of workers: 3 I0521 19:49:09.469323 1 tasks_processing.go:69] worker 2 listening for tasks. I0521 19:49:09.469327 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0521 19:49:09.469340 1 tasks_processing.go:69] worker 0 listening for tasks. I0521 19:49:09.469353 1 tasks_processing.go:69] worker 1 listening for tasks. I0521 19:49:09.469360 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0521 19:49:09.469362 1 tasks_processing.go:74] worker 1 stopped. I0521 19:49:09.469356 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0521 19:49:09.469398 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0521 19:49:09.469409 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.181µs to process 1 records I0521 19:49:09.469439 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0521 19:49:09.469455 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.138µs to process 1 records I0521 19:49:09.469463 1 tasks_processing.go:74] worker 0 stopped. I0521 19:49:09.469610 1 tasks_processing.go:74] worker 2 stopped. I0521 19:49:09.469623 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 244.377µs to process 0 records I0521 19:49:09.469646 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0521 19:49:09.469663 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0521 19:49:09.496613 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=6ff7c501c7358552cd30684bb51882901824b06b79b518c461bf00b077738ead I0521 19:49:09.496733 1 diskrecorder.go:70] Writing 111 records to /var/lib/insights-operator/insights-2026-05-21-194909.tar.gz I0521 19:49:09.503416 1 diskrecorder.go:51] Wrote 111 records to disk in 6ms I0521 19:49:09.503443 1 periodic.go:278] Gathering cluster info every 2h0m0s I0521 19:49:09.503459 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0521 19:50:16.858197 1 diskrecorder.go:223] Found files to send: insights-2026-05-21-194909.tar.gz I0521 19:50:16.858225 1 insightsuploader.go:150] Checking archives to upload periodically every 16m2.67951498s I0521 19:50:16.858233 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0521 19:50:16.869300 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0521 19:50:17.009632 1 requests.go:87] Successfully reported id=2026-05-21T19:50:16Z x-rh-insights-request-id=b58c02cbc32640f7aab99399ce9b43be, wrote=49308 I0521 19:50:17.009685 1 insightsuploader.go:187] Uploaded report successfully in 151.444633ms I0521 19:50:17.009706 1 controller.go:128] Initializing last reported time to 2026-05-21T19:50:16Z I0521 19:50:17.009722 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0521 19:50:17.009734 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0521 19:50:17.009741 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0521 19:50:17.017989 1 controller.go:512] The operator is healthy I0521 19:50:46.884349 1 controller.go:512] The operator is healthy I0521 19:51:21.259170 1 insightsreport.go:137] Pulling report from smart-proxy I0521 19:51:21.259202 1 insightsreport.go:149] Retrieving report I0521 19:51:21.275426 1 requests.go:110] Retrieving report for cluster: 0e710525-dce6-4076-91ef-6f3f27696254 I0521 19:51:21.275438 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/0e710525-dce6-4076-91ef-6f3f27696254/reports I0521 19:51:21.278362 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/0e710525-dce6-4076-91ef-6f3f27696254/reports I0521 19:51:21.399099 1 insightsreport.go:184] Report retrieved I0521 19:51:21.452850 1 insightsreport.go:239] Report retrieved correctly I0521 19:52:46.884734 1 controller.go:512] The operator is healthy I0521 19:53:46.850329 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0521 19:53:46.855970 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 19:53:46.855994 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0521 19:53:46.860075 1 secretconfigobserver.go:119] support secret does not exist I0521 19:54:46.884002 1 controller.go:512] The operator is healthy I0521 19:56:46.884105 1 controller.go:512] The operator is healthy I0521 19:58:46.848189 1 configmapobserver.go:84] configmaps "insights-config" not found I0521 19:58:46.852226 1 configmapobserver.go:84] configmaps "insights-config" not found I0521 19:58:46.860311 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0521 19:58:46.864561 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 19:58:46.864578 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0521 19:58:46.868210 1 secretconfigobserver.go:119] support secret does not exist I0521 19:58:46.883375 1 controller.go:512] The operator is healthy I0521 20:00:46.884077 1 controller.go:512] The operator is healthy I0521 20:02:46.884781 1 controller.go:512] The operator is healthy I0521 20:03:46.868815 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0521 20:03:46.874805 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 20:03:46.874825 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0521 20:03:46.879191 1 secretconfigobserver.go:119] support secret does not exist I0521 20:04:46.886664 1 controller.go:512] The operator is healthy I0521 20:06:24.689580 1 insightsuploader.go:146] Nothing to report since 2026-05-21T19:50:16Z I0521 20:06:46.887321 1 controller.go:512] The operator is healthy I0521 20:08:46.851683 1 configmapobserver.go:84] configmaps "insights-config" not found I0521 20:08:46.879895 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0521 20:08:46.883829 1 controller.go:512] The operator is healthy I0521 20:08:46.884981 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 20:08:46.885007 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0521 20:08:46.889187 1 secretconfigobserver.go:119] support secret does not exist I0521 20:10:46.887218 1 controller.go:512] The operator is healthy I0521 20:12:46.886730 1 controller.go:512] The operator is healthy I0521 20:13:46.890017 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0521 20:13:46.898603 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0521 20:13:46.898625 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0521 20:13:46.904716 1 secretconfigobserver.go:119] support secret does not exist I0521 20:14:46.887059 1 controller.go:512] The operator is healthy