I0401 11:31:00.131403 1 cmd.go:253] Using service-serving-cert provided certificates I0401 11:31:00.131932 1 observer_polling.go:159] Starting file observer I0401 11:31:00.594381 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0401 11:31:00.594569 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0401 11:31:00.596423 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0401 11:31:00.597095 1 secure_serving.go:57] Forcing use of http/1.1 only W0401 11:31:00.597123 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0401 11:31:00.597129 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0401 11:31:00.597135 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0401 11:31:00.597139 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0401 11:31:00.597144 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0401 11:31:00.597148 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0401 11:31:00.599644 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0401 11:31:00.599655 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"db5f5a77-6e73-46cf-9640-9b9652ee0c4b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0401 11:31:00.601380 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0401 11:31:00.601387 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0401 11:31:00.601401 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0401 11:31:00.601393 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0401 11:31:00.601416 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0401 11:31:00.601401 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0401 11:31:00.601627 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0401 11:31:00.601931 1 secure_serving.go:213] Serving securely on [::]:8443 I0401 11:31:00.601959 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0401 11:31:00.604603 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0401 11:31:00.604624 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0401 11:31:00.604736 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0401 11:31:00.609244 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:31:00.609262 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0401 11:31:00.613189 1 secretconfigobserver.go:119] support secret does not exist I0401 11:31:00.617096 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:31:00.620573 1 secretconfigobserver.go:119] support secret does not exist I0401 11:31:00.623273 1 recorder.go:161] Pruning old reports every 4h13m32s, max age is 288h0m0s I0401 11:31:00.627941 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0401 11:31:00.627952 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0401 11:31:00.627956 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0401 11:31:00.627961 1 insightsreport.go:296] Starting report retriever I0401 11:31:00.627969 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0401 11:31:00.628059 1 periodic.go:209] Running clusterconfig gatherer I0401 11:31:00.628341 1 tasks_processing.go:45] number of workers: 64 I0401 11:31:00.628383 1 tasks_processing.go:69] worker 4 listening for tasks. I0401 11:31:00.628403 1 tasks_processing.go:69] worker 0 listening for tasks. I0401 11:31:00.628414 1 tasks_processing.go:69] worker 1 listening for tasks. I0401 11:31:00.628476 1 tasks_processing.go:69] worker 2 listening for tasks. I0401 11:31:00.628480 1 tasks_processing.go:69] worker 6 listening for tasks. I0401 11:31:00.628491 1 tasks_processing.go:69] worker 3 listening for tasks. I0401 11:31:00.628556 1 tasks_processing.go:69] worker 9 listening for tasks. I0401 11:31:00.628551 1 tasks_processing.go:69] worker 5 listening for tasks. I0401 11:31:00.628592 1 tasks_processing.go:69] worker 7 listening for tasks. I0401 11:31:00.628741 1 tasks_processing.go:69] worker 10 listening for tasks. I0401 11:31:00.629168 1 tasks_processing.go:69] worker 8 listening for tasks. I0401 11:31:00.629193 1 tasks_processing.go:69] worker 19 listening for tasks. I0401 11:31:00.629195 1 tasks_processing.go:69] worker 34 listening for tasks. I0401 11:31:00.629200 1 tasks_processing.go:69] worker 32 listening for tasks. I0401 11:31:00.629209 1 tasks_processing.go:69] worker 20 listening for tasks. I0401 11:31:00.629192 1 tasks_processing.go:69] worker 18 listening for tasks. I0401 11:31:00.629215 1 tasks_processing.go:69] worker 33 listening for tasks. I0401 11:31:00.629214 1 tasks_processing.go:69] worker 14 listening for tasks. I0401 11:31:00.629224 1 tasks_processing.go:69] worker 30 listening for tasks. I0401 11:31:00.629224 1 tasks_processing.go:69] worker 28 listening for tasks. I0401 11:31:00.629224 1 tasks_processing.go:69] worker 31 listening for tasks. I0401 11:31:00.629229 1 tasks_processing.go:69] worker 35 listening for tasks. I0401 11:31:00.629236 1 tasks_processing.go:69] worker 25 listening for tasks. I0401 11:31:00.629238 1 tasks_processing.go:69] worker 29 listening for tasks. I0401 11:31:00.629205 1 tasks_processing.go:69] worker 15 listening for tasks. I0401 11:31:00.629246 1 tasks_processing.go:69] worker 41 listening for tasks. I0401 11:31:00.629201 1 tasks_processing.go:69] worker 40 listening for tasks. I0401 11:31:00.629216 1 tasks_processing.go:69] worker 27 listening for tasks. I0401 11:31:00.629217 1 tasks_processing.go:69] worker 16 listening for tasks. I0401 11:31:00.629213 1 tasks_processing.go:69] worker 39 listening for tasks. I0401 11:31:00.629219 1 tasks_processing.go:69] worker 21 listening for tasks. I0401 11:31:00.629228 1 tasks_processing.go:69] worker 22 listening for tasks. I0401 11:31:00.629232 1 tasks_processing.go:69] worker 24 listening for tasks. I0401 11:31:00.629234 1 tasks_processing.go:69] worker 11 listening for tasks. I0401 11:31:00.629243 1 tasks_processing.go:69] worker 37 listening for tasks. I0401 11:31:00.629248 1 tasks_processing.go:69] worker 12 listening for tasks. I0401 11:31:00.629179 1 tasks_processing.go:69] worker 26 listening for tasks. I0401 11:31:00.629322 1 tasks_processing.go:69] worker 42 listening for tasks. I0401 11:31:00.629326 1 tasks_processing.go:69] worker 38 listening for tasks. I0401 11:31:00.629599 1 tasks_processing.go:69] worker 51 listening for tasks. I0401 11:31:00.629604 1 tasks_processing.go:69] worker 13 listening for tasks. I0401 11:31:00.629610 1 tasks_processing.go:69] worker 23 listening for tasks. I0401 11:31:00.629611 1 tasks_processing.go:69] worker 50 listening for tasks. I0401 11:31:00.629612 1 tasks_processing.go:69] worker 17 listening for tasks. I0401 11:31:00.629601 1 tasks_processing.go:69] worker 45 listening for tasks. I0401 11:31:00.629622 1 tasks_processing.go:69] worker 46 listening for tasks. I0401 11:31:00.629597 1 tasks_processing.go:69] worker 52 listening for tasks. I0401 11:31:00.629651 1 tasks_processing.go:69] worker 48 listening for tasks. I0401 11:31:00.629657 1 tasks_processing.go:69] worker 36 listening for tasks. I0401 11:31:00.629664 1 tasks_processing.go:69] worker 53 listening for tasks. I0401 11:31:00.629608 1 tasks_processing.go:69] worker 44 listening for tasks. I0401 11:31:00.629607 1 tasks_processing.go:69] worker 43 listening for tasks. I0401 11:31:00.629625 1 tasks_processing.go:69] worker 47 listening for tasks. I0401 11:31:00.629629 1 tasks_processing.go:69] worker 49 listening for tasks. I0401 11:31:00.629701 1 tasks_processing.go:69] worker 54 listening for tasks. I0401 11:31:00.629717 1 tasks_processing.go:69] worker 57 listening for tasks. I0401 11:31:00.629727 1 tasks_processing.go:69] worker 55 listening for tasks. I0401 11:31:00.629729 1 tasks_processing.go:69] worker 58 listening for tasks. I0401 11:31:00.629733 1 tasks_processing.go:69] worker 56 listening for tasks. I0401 11:31:00.629737 1 tasks_processing.go:69] worker 59 listening for tasks. I0401 11:31:00.629736 1 tasks_processing.go:69] worker 60 listening for tasks. I0401 11:31:00.630019 1 tasks_processing.go:69] worker 61 listening for tasks. I0401 11:31:00.630029 1 tasks_processing.go:71] worker 4 working on dvo_metrics task. I0401 11:31:00.630035 1 tasks_processing.go:71] worker 30 working on container_runtime_configs task. I0401 11:31:00.630036 1 tasks_processing.go:71] worker 14 working on openstack_dataplanedeployments task. I0401 11:31:00.630040 1 tasks_processing.go:71] worker 61 working on clusterroles task. I0401 11:31:00.630043 1 tasks_processing.go:69] worker 62 listening for tasks. I0401 11:31:00.630055 1 tasks_processing.go:71] worker 62 working on ingress task. I0401 11:31:00.630063 1 tasks_processing.go:71] worker 39 working on certificate_signing_requests task. I0401 11:31:00.630068 1 tasks_processing.go:71] worker 2 working on machine_autoscalers task. I0401 11:31:00.630071 1 tasks_processing.go:71] worker 29 working on aggregated_monitoring_cr_names task. I0401 11:31:00.630080 1 tasks_processing.go:71] worker 41 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0401 11:31:00.630096 1 tasks_processing.go:71] worker 31 working on sap_datahubs task. I0401 11:31:00.630111 1 tasks_processing.go:71] worker 47 working on version task. I0401 11:31:00.630117 1 tasks_processing.go:71] worker 28 working on oauths task. I0401 11:31:00.630131 1 tasks_processing.go:71] worker 26 working on ceph_cluster task. I0401 11:31:00.630139 1 tasks_processing.go:71] worker 49 working on jaegers task. I0401 11:31:00.630144 1 tasks_processing.go:69] worker 63 listening for tasks. I0401 11:31:00.630153 1 tasks_processing.go:71] worker 13 working on support_secret task. I0401 11:31:00.630159 1 tasks_processing.go:71] worker 6 working on networks task. I0401 11:31:00.630176 1 tasks_processing.go:71] worker 32 working on machine_configs task. I0401 11:31:00.630184 1 tasks_processing.go:71] worker 21 working on openstack_dataplanenodesets task. I0401 11:31:00.630206 1 tasks_processing.go:71] worker 50 working on storage_cluster task. I0401 11:31:00.630268 1 tasks_processing.go:71] worker 35 working on image_registries task. I0401 11:31:00.630282 1 tasks_processing.go:71] worker 40 working on pdbs task. I0401 11:31:00.630103 1 tasks_processing.go:71] worker 15 working on operators_pods_and_events task. I0401 11:31:00.630312 1 tasks_processing.go:71] worker 23 working on validating_webhook_configurations task. I0401 11:31:00.630325 1 tasks_processing.go:71] worker 9 working on monitoring_persistent_volumes task. I0401 11:31:00.630328 1 tasks_processing.go:71] worker 18 working on qemu_kubevirt_launcher_logs task. I0401 11:31:00.630336 1 tasks_processing.go:71] worker 46 working on operators task. I0401 11:31:00.630343 1 tasks_processing.go:71] worker 54 working on infrastructures task. I0401 11:31:00.630363 1 tasks_processing.go:71] worker 5 working on node_logs task. I0401 11:31:00.630323 1 tasks_processing.go:71] worker 20 working on nodenetworkstates task. I0401 11:31:00.630379 1 tasks_processing.go:71] worker 22 working on pod_network_connectivity_checks task. I0401 11:31:00.630391 1 tasks_processing.go:71] worker 44 working on storage_classes task. I0401 11:31:00.630436 1 tasks_processing.go:71] worker 27 working on cluster_apiserver task. I0401 11:31:00.630459 1 tasks_processing.go:71] worker 19 working on machine_sets task. I0401 11:31:00.630493 1 tasks_processing.go:71] worker 57 working on silenced_alerts task. I0401 11:31:00.630506 1 tasks_processing.go:71] worker 42 working on nodes task. I0401 11:31:00.630331 1 tasks_processing.go:71] worker 36 working on metrics task. I0401 11:31:00.630549 1 tasks_processing.go:71] worker 17 working on lokistack task. I0401 11:31:00.630559 1 tasks_processing.go:71] worker 33 working on authentication task. I0401 11:31:00.630565 1 tasks_processing.go:71] worker 38 working on olm_operators task. I0401 11:31:00.630576 1 tasks_processing.go:71] worker 25 working on openstack_version task. I0401 11:31:00.630584 1 tasks_processing.go:71] worker 52 working on nodenetworkconfigurationpolicies task. I0401 11:31:00.630063 1 tasks_processing.go:71] worker 1 working on openstack_controlplanes task. I0401 11:31:00.630609 1 tasks_processing.go:71] worker 43 working on install_plans task. I0401 11:31:00.630617 1 tasks_processing.go:71] worker 53 working on crds task. I0401 11:31:00.630647 1 tasks_processing.go:71] worker 7 working on cost_management_metrics_configs task. I0401 11:31:00.630656 1 tasks_processing.go:71] worker 34 working on feature_gates task. I0401 11:31:00.630057 1 tasks_processing.go:71] worker 0 working on mutating_webhook_configurations task. I0401 11:31:00.630031 1 tasks_processing.go:71] worker 16 working on image task. I0401 11:31:00.630321 1 tasks_processing.go:71] worker 3 working on machine_config_pools task. I0401 11:31:00.630152 1 tasks_processing.go:71] worker 8 working on service_accounts task. I0401 11:31:00.630174 1 tasks_processing.go:71] worker 24 working on schedulers task. I0401 11:31:00.630745 1 tasks_processing.go:71] worker 48 working on proxies task. I0401 11:31:00.630774 1 tasks_processing.go:71] worker 37 working on openshift_machine_api_events task. I0401 11:31:00.630779 1 tasks_processing.go:71] worker 11 working on container_images task. I0401 11:31:00.630059 1 tasks_processing.go:71] worker 45 working on sap_config task. I0401 11:31:00.630783 1 tasks_processing.go:71] worker 12 working on image_pruners task. I0401 11:31:00.630786 1 tasks_processing.go:71] worker 63 working on machine_healthchecks task. I0401 11:31:00.630792 1 tasks_processing.go:71] worker 59 working on config_maps task. I0401 11:31:00.630791 1 tasks_processing.go:71] worker 10 working on openshift_logging task. I0401 11:31:00.630791 1 tasks_processing.go:71] worker 58 working on tsdb_status task. I0401 11:31:00.630796 1 tasks_processing.go:71] worker 55 working on machines task. I0401 11:31:00.630798 1 tasks_processing.go:71] worker 56 working on sap_pods task. I0401 11:31:00.630803 1 tasks_processing.go:71] worker 60 working on active_alerts task. I0401 11:31:00.630737 1 tasks_processing.go:71] worker 51 working on ingress_certificates task. I0401 11:31:00.633318 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0401 11:31:00.633337 1 controller.go:326] The initial operator extension status is healthy I0401 11:31:00.633348 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0401 11:31:00.633352 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0401 11:31:00.633355 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0401 11:31:00.633368 1 controller.go:489] The operator is still being initialized I0401 11:31:00.633373 1 controller.go:512] The operator is healthy I0401 11:31:00.633452 1 tasks_processing.go:71] worker 31 working on overlapping_namespace_uids task. I0401 11:31:00.633456 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 3.334596ms to process 0 records I0401 11:31:00.633466 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0401 11:31:00.633483 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0401 11:31:00.633585 1 operator.go:288] started I0401 11:31:00.633609 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer E0401 11:31:00.636464 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:60078->172.30.0.10:53: read: connection refused I0401 11:31:00.636479 1 tasks_processing.go:74] worker 60 stopped. E0401 11:31:00.636489 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:60078->172.30.0.10:53: read: connection refused I0401 11:31:00.636499 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 4.744576ms to process 0 records E0401 11:31:00.636503 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:60078->172.30.0.10:53: read: connection refused I0401 11:31:00.636522 1 tasks_processing.go:74] worker 57 stopped. E0401 11:31:00.636533 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:60078->172.30.0.10:53: read: connection refused I0401 11:31:00.636543 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 6.011211ms to process 0 records E0401 11:31:00.636554 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:56169->172.30.0.10:53: read: connection refused I0401 11:31:00.636571 1 tasks_processing.go:74] worker 58 stopped. E0401 11:31:00.636577 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:56169->172.30.0.10:53: read: connection refused I0401 11:31:00.636586 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.999684ms to process 0 records E0401 11:31:00.636882 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:56169->172.30.0.10:53: read: connection refused I0401 11:31:00.636927 1 tasks_processing.go:74] worker 36 stopped. E0401 11:31:00.636938 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.17:56169->172.30.0.10:53: read: connection refused I0401 11:31:00.636962 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.373244ms to process 0 records I0401 11:31:00.646969 1 tasks_processing.go:74] worker 30 stopped. I0401 11:31:00.646986 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 16.924396ms to process 0 records I0401 11:31:00.647009 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 16.499256ms to process 0 records I0401 11:31:00.647015 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 16.785909ms to process 0 records I0401 11:31:00.647020 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 16.766077ms to process 0 records I0401 11:31:00.647020 1 tasks_processing.go:74] worker 19 stopped. I0401 11:31:00.647027 1 tasks_processing.go:74] worker 49 stopped. I0401 11:31:00.647023 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 16.841348ms to process 0 records I0401 11:31:00.647034 1 tasks_processing.go:74] worker 50 stopped. I0401 11:31:00.647044 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 16.934292ms to process 0 records I0401 11:31:00.647052 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 16.964181ms to process 0 records I0401 11:31:00.647059 1 tasks_processing.go:74] worker 14 stopped. I0401 11:31:00.647032 1 tasks_processing.go:74] worker 21 stopped. I0401 11:31:00.647066 1 tasks_processing.go:74] worker 2 stopped. I0401 11:31:00.647735 1 tasks_processing.go:74] worker 62 stopped. I0401 11:31:00.647936 1 recorder.go:75] Recording config/ingress with fingerprint=77d606ed8a385cc72dcc8bcd408e14f879b40dab1035e3ed5924ff698580315f I0401 11:31:00.647949 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 17.662828ms to process 1 records I0401 11:31:00.649330 1 tasks_processing.go:74] worker 52 stopped. I0401 11:31:00.649345 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 18.73064ms to process 0 records I0401 11:31:00.649435 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=708f8de59c1ea6522f093bab0274413564678fd01bddda560305fb05fcbeaf4b I0401 11:31:00.649447 1 tasks_processing.go:74] worker 44 stopped. I0401 11:31:00.649452 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=0a7f26ff311683680f656c44f973b3bdeb273cfa242b594c3e06bcf18c2356b5 I0401 11:31:00.649458 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 18.945173ms to process 2 records I0401 11:31:00.654842 1 tasks_processing.go:74] worker 13 stopped. E0401 11:31:00.654853 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0401 11:31:00.654860 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 24.681909ms to process 0 records I0401 11:31:00.654873 1 tasks_processing.go:74] worker 1 stopped. I0401 11:31:00.654883 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 24.260725ms to process 0 records E0401 11:31:00.654893 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0401 11:31:00.654898 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 24.487624ms to process 0 records I0401 11:31:00.654902 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 24.491416ms to process 0 records I0401 11:31:00.654916 1 tasks_processing.go:74] worker 20 stopped. I0401 11:31:00.654923 1 tasks_processing.go:74] worker 22 stopped. I0401 11:31:00.655047 1 tasks_processing.go:74] worker 7 stopped. I0401 11:31:00.655061 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 24.364047ms to process 0 records I0401 11:31:00.655266 1 tasks_processing.go:74] worker 6 stopped. I0401 11:31:00.655413 1 recorder.go:75] Recording config/network with fingerprint=63b192cb96cc73064093dbf08fc0f0ad3b2590464d03494f7291a474b3443a3c I0401 11:31:00.655426 1 gather.go:177] gatherer "clusterconfig" function "networks" took 25.091449ms to process 1 records I0401 11:31:00.655511 1 tasks_processing.go:74] worker 40 stopped. I0401 11:31:00.655608 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=68a1e226d4682856f19528a3727c9b5d9ccba51687f59698edfad0aa5de5741d I0401 11:31:00.655627 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=f76f316824043911156ef208e9ac81c55a6eaefc5270cfb12ca9aa1964d4547f I0401 11:31:00.655641 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=e3d575951977208301448ebf4a6d6663cbb99ef0c2755881bcd4d71933c49e8b I0401 11:31:00.655655 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=f8fbeaa1a712e31f4ddc9ca28868cd40872e0fcf485532886351e988b06ae687 I0401 11:31:00.655693 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=6b4f6d8e0a900f32f358aa5ba7a68a315ba9df8fb5b8e6de3482bc70ba8e8701 I0401 11:31:00.655714 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=813047896361da48d02092ca2af3da23702b4b0391f4e360e8e559619fb59ecf I0401 11:31:00.655720 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 25.219178ms to process 6 records I0401 11:31:00.655726 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 25.057095ms to process 0 records I0401 11:31:00.655732 1 tasks_processing.go:74] worker 25 stopped. I0401 11:31:00.655813 1 tasks_processing.go:74] worker 27 stopped. I0401 11:31:00.655956 1 recorder.go:75] Recording config/apiserver with fingerprint=4f9f59003dad8a1be286427df672b86e4c27427d7a56e8699eb0d87e4da5e7ba I0401 11:31:00.655969 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 25.363583ms to process 1 records I0401 11:31:00.656003 1 tasks_processing.go:74] worker 9 stopped. I0401 11:31:00.656022 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 25.650446ms to process 0 records I0401 11:31:00.656056 1 tasks_processing.go:74] worker 3 stopped. I0401 11:31:00.656068 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 25.164133ms to process 0 records I0401 11:31:00.656086 1 tasks_processing.go:74] worker 16 stopped. I0401 11:31:00.656155 1 recorder.go:75] Recording config/image with fingerprint=b91185981f0b27053f9382c0e9e0669762726ef8949f06e83e9149ece2696076 I0401 11:31:00.656166 1 gather.go:177] gatherer "clusterconfig" function "image" took 25.270134ms to process 1 records I0401 11:31:00.656200 1 tasks_processing.go:74] worker 54 stopped. I0401 11:31:00.656737 1 recorder.go:75] Recording config/infrastructure with fingerprint=58f0caeb06b4dc1a26212686a8c255a137b97a612920f996d6778be6c7a9ee58 I0401 11:31:00.656750 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 25.844747ms to process 1 records I0401 11:31:00.656828 1 tasks_processing.go:74] worker 33 stopped. I0401 11:31:00.656913 1 recorder.go:75] Recording config/authentication with fingerprint=563c3799ba149289cc3a1870a3bdd94c000796558070d51bcdc9e90240b4b55b I0401 11:31:00.656922 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 25.680879ms to process 1 records I0401 11:31:00.656927 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 24.860035ms to process 0 records I0401 11:31:00.656940 1 tasks_processing.go:74] worker 56 stopped. I0401 11:31:00.657022 1 gather_logs.go:145] no pods in namespace were found I0401 11:31:00.657074 1 tasks_processing.go:74] worker 28 stopped. I0401 11:31:00.657151 1 recorder.go:75] Recording config/oauth with fingerprint=0b2e3f5cf055d74dea2bf38db783a8dbbed65be351817fd8ebd0f3b3e153e853 I0401 11:31:00.657164 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 26.714106ms to process 1 records I0401 11:31:00.657174 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 26.42267ms to process 0 records I0401 11:31:00.657180 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 26.609352ms to process 0 records I0401 11:31:00.657186 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 26.693094ms to process 0 records I0401 11:31:00.657188 1 tasks_processing.go:74] worker 5 stopped. I0401 11:31:00.657192 1 tasks_processing.go:74] worker 17 stopped. I0401 11:31:00.657198 1 tasks_processing.go:74] worker 18 stopped. I0401 11:31:00.657236 1 recorder.go:75] Recording config/proxy with fingerprint=69eca861860911f2e727c1032aaf0c70ef989e6afbcb3916b21fb3fe155d72df I0401 11:31:00.657248 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 25.926341ms to process 1 records I0401 11:31:00.657269 1 tasks_processing.go:74] worker 48 stopped. I0401 11:31:00.657312 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=5ed78fc241da0e713afbfc70293d0c7527da7430195701502d9409b5fa8e239c I0401 11:31:00.657324 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 25.976384ms to process 1 records I0401 11:31:00.657332 1 tasks_processing.go:74] worker 24 stopped. I0401 11:31:00.658360 1 tasks_processing.go:74] worker 39 stopped. I0401 11:31:00.658374 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 28.283114ms to process 0 records I0401 11:31:00.658706 1 tasks_processing.go:74] worker 10 stopped. I0401 11:31:00.658719 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 27.164073ms to process 0 records I0401 11:31:00.658803 1 tasks_processing.go:74] worker 12 stopped. I0401 11:31:00.658933 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=1f7fe3c3d735c0f4c6ed7e6240abab4106b9e6b1b76b352379011c51c62ce60a I0401 11:31:00.658950 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 27.308ms to process 1 records I0401 11:31:00.658963 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 27.359335ms to process 0 records E0401 11:31:00.658969 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0401 11:31:00.658975 1 gather.go:177] gatherer "clusterconfig" function "machines" took 27.052259ms to process 0 records I0401 11:31:00.658981 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 28.594404ms to process 0 records E0401 11:31:00.658985 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0401 11:31:00.659007 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 27.321573ms to process 0 records I0401 11:31:00.659013 1 tasks_processing.go:74] worker 55 stopped. I0401 11:31:00.659020 1 tasks_processing.go:74] worker 45 stopped. I0401 11:31:00.659038 1 tasks_processing.go:74] worker 26 stopped. I0401 11:31:00.659041 1 tasks_processing.go:74] worker 63 stopped. I0401 11:31:00.659096 1 recorder.go:75] Recording config/featuregate with fingerprint=40710f819e1e23965a1779ffd2c0f7ff737b531764b0756a107b13247c8a042e I0401 11:31:00.659106 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 28.265947ms to process 1 records I0401 11:31:00.659108 1 tasks_processing.go:74] worker 34 stopped. I0401 11:31:00.659675 1 tasks_processing.go:74] worker 0 stopped. I0401 11:31:00.659880 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=fe5e366965119e8d58cc03e7840ee106b9670a5d081f967ee44fa123133dc017 I0401 11:31:00.659935 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=14849af43412eca0b505eeea52cd135692d220976a710a8ecefbc1e132af21b2 I0401 11:31:00.659981 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=e1a248f50274f216fc6f34ff77109296d211a08ba38a46c411c50167864449b8 I0401 11:31:00.660008 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 28.897018ms to process 3 records I0401 11:31:00.660099 1 tasks_processing.go:74] worker 35 stopped. I0401 11:31:00.662090 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=cfbf936bcb884ca5138fb21d7309b4e02298026863e3f7dbdbcd159ae3e77302 I0401 11:31:00.662296 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 29.670918ms to process 1 records I0401 11:31:00.662314 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 28.882857ms to process 0 records I0401 11:31:00.662321 1 tasks_processing.go:74] worker 37 stopped. W0401 11:31:00.662560 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0401 11:31:00.663272 1 tasks_processing.go:74] worker 42 stopped. I0401 11:31:00.663313 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0401 11:31:00.663329 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0401 11:31:00.663334 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0401 11:31:00.663338 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0401 11:31:00.663343 1 controller.go:212] Source scaController *sca.Controller is not ready I0401 11:31:00.663380 1 controller.go:489] The operator is still being initialized I0401 11:31:00.663398 1 controller.go:512] The operator is healthy I0401 11:31:00.663560 1 recorder.go:75] Recording config/node/ip-10-0-0-170.ec2.internal with fingerprint=793129627b131ab0398b726451a2e0295be0737ae1d5b8a86fa137956cc8ec24 I0401 11:31:00.663614 1 recorder.go:75] Recording config/node/ip-10-0-1-27.ec2.internal with fingerprint=e350c93f5ea47f46442b3b17096b5e7c88c9f0be420b6740cdbc8c50baceff2b I0401 11:31:00.663623 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 32.757129ms to process 2 records I0401 11:31:00.664437 1 tasks_processing.go:74] worker 31 stopped. I0401 11:31:00.664463 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0401 11:31:00.664472 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 30.972171ms to process 1 records I0401 11:31:00.664911 1 tasks_processing.go:74] worker 23 stopped. I0401 11:31:00.665114 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=d796912fc9a1d529ea22bd324a81ffdfa315cfd9594d71a0b93f611e97a26b61 I0401 11:31:00.665268 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=f4bee73ef66d31ad100cab1ff6b8351cc4dd60682b004551cda2e5ef697fd2ae I0401 11:31:00.665319 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=d7068d3343421bf2bca365824d436b1ae122f346b501cc7b5f5c09e1dbeae7a0 I0401 11:31:00.665380 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=90357ea449c099b6cdcebfb997887689f2a8453b1c59bf98ca4d7c60ab4ff962 I0401 11:31:00.665431 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=7cfa6c24029c6aa71fbc07c1a7df46b4218878ec2ae542f23e3b35ad5f0e447d I0401 11:31:00.665480 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=6f1510eba7152d6e692e0291c3ba706319f44f3a15e6f18135d230f6b3bd8869 I0401 11:31:00.665529 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=df15b4c30dc90aa7d2052f7cb394851e3958467f2bf417a255f8877e635aa175 I0401 11:31:00.665600 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=af0e1977c533a5402655ea0776fdaf61b7e9556038c83b7164298d10bd4b2f2a I0401 11:31:00.665649 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=64b38e75720e5ac115abd68e3c5134f971131e1a0dfcc6b1a734109f3d8349ac I0401 11:31:00.665698 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=3b558c40d98d08f8cf95460e7b99a77cc9adfbc694eed3c8431eee76ca1b6554 I0401 11:31:00.665767 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=8b2aecc5c84f064c836c3b5cad945473b16cafccb44ddc0e0edbdf4975a5b634 I0401 11:31:00.665782 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 34.57239ms to process 11 records I0401 11:31:00.669731 1 tasks_processing.go:74] worker 47 stopped. I0401 11:31:00.670123 1 recorder.go:75] Recording config/version with fingerprint=67b05dbc47f58134a2bd29324cb0b006397cefb6fd502fd31d3f1cde245e8129 I0401 11:31:00.670142 1 recorder.go:75] Recording config/id with fingerprint=3aa586145dab1e59324ad96c2b43eb39a2f87ea204de2337da4d0546f67b7db9 I0401 11:31:00.670151 1 gather.go:177] gatherer "clusterconfig" function "version" took 39.607108ms to process 2 records I0401 11:31:00.670235 1 tasks_processing.go:74] worker 61 stopped. I0401 11:31:00.670412 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=98a1904c6240d783a4d110602d05c1ab903badd7983da280ecc8d4b1670f6aaa I0401 11:31:00.670528 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=721c6f424aeb3359e1148cc368531742cd6369399d5d09f33fedd01fefc2ad8b I0401 11:31:00.670570 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 39.970361ms to process 2 records I0401 11:31:00.671462 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=1ca1f0b03582e8bc88758cc3e0044794db4c5bde6688ed6d2b7709fb58d034eb I0401 11:31:00.671478 1 tasks_processing.go:74] worker 53 stopped. I0401 11:31:00.671771 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=f64669a8b364661df4b98a70a45d6b2869f7626105f36ef8ddbaaffb74f6f384 I0401 11:31:00.671786 1 gather.go:177] gatherer "clusterconfig" function "crds" took 39.494184ms to process 2 records I0401 11:31:00.671801 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 40.161315ms to process 0 records I0401 11:31:00.671837 1 tasks_processing.go:74] worker 29 stopped. I0401 11:31:00.671867 1 recorder.go:75] Recording config/olm_operators with fingerprint=48d82a578b56747f6bcfdfb674f39129a416ef8569b0a4137c4c958edcc9547b I0401 11:31:00.671876 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 39.934135ms to process 1 records I0401 11:31:00.671885 1 tasks_processing.go:74] worker 38 stopped. I0401 11:31:00.674539 1 tasks_processing.go:74] worker 32 stopped. I0401 11:31:00.674564 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0401 11:31:00.674574 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 44.348876ms to process 1 records I0401 11:31:00.676722 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0401 11:31:00.676870 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.17:34221->172.30.0.10:53: read: connection refused I0401 11:31:00.676888 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.17:34221->172.30.0.10:53: read: connection refused E0401 11:31:00.676877 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27027222a6-0fff-45b3-90d9-4d42ebdc4d30%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.17:34221->172.30.0.10:53: read: connection refused I0401 11:31:00.676900 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27027222a6-0fff-45b3-90d9-4d42ebdc4d30%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.17:34221->172.30.0.10:53: read: connection refused I0401 11:31:00.683556 1 tasks_processing.go:74] worker 11 stopped. I0401 11:31:00.685325 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf with fingerprint=62d2fda53b1f5254544a6e3eae99e4f4637cbdad6c330b76accf0e714476b0e4 I0401 11:31:00.685651 1 recorder.go:75] Recording config/pod/openshift-multus/multus-2xdcm with fingerprint=41ec056b5a3058d21a30ef910af81a419d385a7e6b6a76791742c2782fa3f87c I0401 11:31:00.686473 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-sbnpw with fingerprint=5346e3120a0fb64d5754c924f33e07cf9345a18cdea66ee0c4202fffc706dfeb I0401 11:31:00.686940 1 recorder.go:75] Recording config/running_containers with fingerprint=1e385d7221b823580341e1f4e5cde048ffe99802f71b4187fd14ce729057813e I0401 11:31:00.686965 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 52.256577ms to process 4 records I0401 11:31:00.686980 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 55.087915ms to process 0 records E0401 11:31:00.687013 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found I0401 11:31:00.687012 1 tasks_processing.go:74] worker 41 stopped. E0401 11:31:00.687020 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found I0401 11:31:00.687025 1 tasks_processing.go:74] worker 59 stopped. E0401 11:31:00.687034 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0401 11:31:00.687047 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0401 11:31:00.687075 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0401 11:31:00.687083 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0401 11:31:00.687088 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0401 11:31:00.687096 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0401 11:31:00.687134 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0401 11:31:00.687141 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0401 11:31:00.687147 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 55.484538ms to process 7 records I0401 11:31:00.691143 1 tasks_processing.go:74] worker 51 stopped. E0401 11:31:00.691158 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0401 11:31:00.691164 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pdart7vjjvdlq4hsmad4q107bvuvhah-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pdart7vjjvdlq4hsmad4q107bvuvhah-primary-cert-bundle-secret" not found I0401 11:31:00.691212 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=2fcb4261fa779f14f8d2b1ecde62957c248137bf41a26445f30272977d6cdffd I0401 11:31:00.691237 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 59.379885ms to process 1 records I0401 11:31:00.701760 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0401 11:31:00.701778 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0401 11:31:00.701793 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0401 11:31:00.705024 1 base_controller.go:82] Caches are synced for ConfigController I0401 11:31:00.705043 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0401 11:31:00.709202 1 configmapobserver.go:84] configmaps "insights-config" not found I0401 11:31:00.734677 1 base_controller.go:82] Caches are synced for LoggingSyncer I0401 11:31:00.734694 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0401 11:31:01.661662 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0401 11:31:02.275474 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 21 containers I0401 11:31:02.275489 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0401 11:31:02.275691 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-fzscz pod in namespace openshift-network-console (previous: false). I0401 11:31:02.479779 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-wtd8q pod in namespace openshift-network-console (previous: false). I0401 11:31:02.488166 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0401 11:31:02.660902 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0401 11:31:02.678314 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-vc9cf pod in namespace openshift-console-operator (previous: true). I0401 11:31:02.878781 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-vc9cf pod in namespace openshift-console-operator (previous: false). I0401 11:31:03.080615 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-5fc79f9bdb-ntc2v pod in namespace openshift-console (previous: false). I0401 11:31:03.091631 1 tasks_processing.go:74] worker 46 stopped. I0401 11:31:03.091749 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=c7e7843ba69d1b29b7c2be77927a14fdf9ebb3e6adfb9b79aeefb9f4a95afe91 I0401 11:31:03.091795 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=b1d3a57201f472fba248e6e22732d03f4ebac77f3d7204455295e9c6f31f376a I0401 11:31:03.091821 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=14ff7e65bfa11fc3d9afef22617507224eb9d65e3ba1dfbde830024a1668d9c5 I0401 11:31:03.091831 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0401 11:31:03.091854 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=76c3f25c5ac6c79fd06f1711a0061c31de11ba55342af9804b8015096582869d I0401 11:31:03.091870 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0401 11:31:03.091893 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=fee38a4a5e8052a2d17344c59c8819e15f66d008e4457f1da379278ccb4703d0 I0401 11:31:03.091922 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=2771061ff2d097bc44afdde1bcd09a34d6759eb3d7c631ae81614c6d92507180 I0401 11:31:03.091945 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=d6d09cf88b3b69e9a849ed12d7e8a5e1b83de0b9c38ed903f5d4e73c9e863a5c I0401 11:31:03.091980 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=9e22d8143c8bf9a2b8a3517550f16df938c49b996c4f40c6c5205679e0e8d58f I0401 11:31:03.092301 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=9143ef06c6a9c76277bc5c04bb9dd7e21f2939879e38bc8303e971c39fd0fa0d I0401 11:31:03.092326 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=0a932b9ac2f86165574ac3459c4abfd62c8d44af986a4d1743432aa3b21920c7 I0401 11:31:03.092337 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0401 11:31:03.092355 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=aa20188af339d76e7a1303dee38f13e9fb1355580a02cf378c9f723d9fd14c77 I0401 11:31:03.092365 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0401 11:31:03.092382 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=c4df3123a8283af622682dcd9d8936049fed25c79ed0eb6c33ac8ec59d773bb6 I0401 11:31:03.092391 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0401 11:31:03.092411 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=3cd818bfae00d231427a84dbb778ca271f1b77012702a4ab0ffc31fae99b83f4 I0401 11:31:03.092421 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0401 11:31:03.092440 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=464a4d3865a33140bd555de91b5a7eaaee05b058aa2a40f472dbcc46d92b724a I0401 11:31:03.092544 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=0a67659b1515904c1fc2287d85a95c4e6e7dd8d7d2d7ebb3f2cf7176a367592e I0401 11:31:03.092554 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0401 11:31:03.092561 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0401 11:31:03.092582 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0401 11:31:03.092605 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=062b506a0734e85ea3e8cbc83c5daded575dcdc414f579acba652cc06ae60a19 I0401 11:31:03.092630 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=5b39fdb49f684d19a93cb19aa1eb0f3941c54a368ce2dae3e4bd728bf6d9d30f I0401 11:31:03.092639 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0401 11:31:03.092657 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=ad3ba58b3e7e9e94656eb6843be1a90fbbba66f08831ed218d872901c0211b13 I0401 11:31:03.092667 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0401 11:31:03.092680 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=95c36dbc1ca21e3f1da749fa5843ca841cebac856a7b555963de0b0d04ff8a5a I0401 11:31:03.092695 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=660af109ab4c93dda52f393fb1177a3d94d24cc76d5c52af76f25e5512ccf595 I0401 11:31:03.092710 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=345c7d7652a404928a0488f34339b25e7eb0347b655778537561b196c59f2248 I0401 11:31:03.092725 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=7849816061d91bf87596d937500ceb4fa78796b69e2d7f16e7b6e7af7b462997 I0401 11:31:03.092748 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=6f78ae5c9e17e88ead04b31e691614da2e2d3da73157dd51c1316ca2778f1199 I0401 11:31:03.092761 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0401 11:31:03.092801 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=98e89e5e1432a2fd26fe19166de0220574379a666220f1f81424869d09f08a97 I0401 11:31:03.092820 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0401 11:31:03.092828 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0401 11:31:03.092835 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.461276213s to process 38 records I0401 11:31:03.274094 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:03.274112 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-5fc79f9bdb-rc2hj pod in namespace openshift-console (previous: false). I0401 11:31:03.472984 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:03.473016 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-6b75f74884-9fsr5 pod in namespace openshift-console (previous: false). W0401 11:31:03.661400 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0401 11:31:03.678647 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-mxpnt pod in namespace openshift-console (previous: false). I0401 11:31:03.877934 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-zmpgm pod in namespace openshift-console (previous: false). I0401 11:31:04.073275 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:04.073295 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-l6nls pod in namespace openshift-dns (previous: false). I0401 11:31:04.281390 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-l6nls pod in namespace openshift-dns (previous: false). I0401 11:31:04.478957 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-f6sr9 pod in namespace openshift-dns (previous: false). W0401 11:31:04.661181 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0401 11:31:04.677586 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:04.677602 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-fkrdq pod in namespace openshift-dns (previous: false). I0401 11:31:04.878815 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:04.878834 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7cc5d98869-7vpwb pod in namespace openshift-image-registry (previous: false). I0401 11:31:05.072935 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:05.072953 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7cc5d98869-9jxfv pod in namespace openshift-image-registry (previous: false). I0401 11:31:05.273155 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:05.273174 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-95bdf696d-qxkl2 pod in namespace openshift-image-registry (previous: false). I0401 11:31:05.478346 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-ccqzp pod in namespace openshift-image-registry (previous: false). I0401 11:31:05.664592 1 tasks_processing.go:74] worker 4 stopped. I0401 11:31:05.664612 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0401 11:31:05.664623 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.034544293s to process 1 records I0401 11:31:05.677986 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-gmxhs pod in namespace openshift-image-registry (previous: false). I0401 11:31:05.878926 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7656455898-gbzn4 pod in namespace openshift-ingress (previous: false). I0401 11:31:06.073141 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0401 11:31:06.073159 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7656455898-rmxff pod in namespace openshift-ingress (previous: false). I0401 11:31:06.279212 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-79675977c5-ss5g4 pod in namespace openshift-ingress (previous: false). I0401 11:31:06.478911 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-sd4lc pod in namespace openshift-ingress-canary (previous: false). I0401 11:31:06.678609 1 tasks_processing.go:74] worker 15 stopped. I0401 11:31:06.678730 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=70860d72efeb85e1ae36aeac3bb7bb0bd0b1ec4b1c909d3f2b5ef4fa248f8480 I0401 11:31:06.678836 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=135e3159d0bb52aef6c7b2729411f6224abd365dbc9e7fed30475a9cd4158b0e I0401 11:31:06.678888 1 recorder.go:75] Recording events/openshift-console with fingerprint=aeca2c26eb228c960ffd6a6b698a9d30f47591b587bf8213b0e9a0db5031260b I0401 11:31:06.678919 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=64c40f8e925b628239d229902c6feed8a2e844bffa37a1f6f945849060619759 I0401 11:31:06.678957 1 recorder.go:75] Recording events/openshift-dns with fingerprint=4b126bf923d6394a6e7d920eb71290198670e856b27003de2e94e4224033d2cf I0401 11:31:06.679065 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=b4b142694183cb59be559c98d338d8ffc7fb054a609ff5781e1a8b3a6436875e I0401 11:31:06.679091 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=0e43f28a630bab987831143510c7fdfb1716fbe984a2ab5821413ba3bfce0b06 I0401 11:31:06.679154 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=1226d185ae92ea91cc563b73a37269e3bd062ec107d0774491b380a0943731b0 I0401 11:31:06.679174 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=3b1dc8ff3eadb10ab024ca1342638e105d4c5e51b7849c0d0bf391f02d528a73 I0401 11:31:06.679183 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-fzscz/networking-console-plugin_current.log with fingerprint=881681a8e508f1534b18c63517824188ee95228369aa877a7761932cd9a2358f I0401 11:31:06.679190 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-wtd8q/networking-console-plugin_current.log with fingerprint=307f91e392bb0ac6814ae10e222e0f26030941c83996801d4711848129e1863c I0401 11:31:06.679321 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf with fingerprint=62d2fda53b1f5254544a6e3eae99e4f4637cbdad6c330b76accf0e714476b0e4 E0401 11:31:06.679337 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json" was already recorded and had the fingerprint "62d2fda53b1f5254544a6e3eae99e4f4637cbdad6c330b76accf0e714476b0e4", overwriting with the record having fingerprint "62d2fda53b1f5254544a6e3eae99e4f4637cbdad6c330b76accf0e714476b0e4" W0401 11:31:06.679347 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json" because of the warning: warning: the record with the same fingerprint "62d2fda53b1f5254544a6e3eae99e4f4637cbdad6c330b76accf0e714476b0e4" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json" I0401 11:31:06.679364 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-vc9cf/console-operator_previous.log with fingerprint=b649c73cd83c1b542a70c4abb28f54a4f4273614d2f435ea63cbbeedd56a0719 I0401 11:31:06.679497 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-vc9cf/console-operator_current.log with fingerprint=db7160afabb1e15b59c46c34b8cef7f6f4db9bd9b95876d71e6e1558fcc9f395 I0401 11:31:06.679510 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-6b75f74884-9fsr5/console_current.log with fingerprint=05f6d638b165e18586faded2a68d2ac7197d3a55f5fc1982bff76c4867a80360 I0401 11:31:06.679516 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-mxpnt/download-server_current.log with fingerprint=be5e8db92b5f802b2b1e332c21b18f4cfaebfa5bae1e0d0defdcc10df5d12de2 I0401 11:31:06.679521 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-l6nls/dns_current.log with fingerprint=310a44eb3bd65a7116cf167e2a05d156a9f9ce62860b6dac622d75fd177ea0e4 I0401 11:31:06.679528 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-l6nls/kube-rbac-proxy_current.log with fingerprint=c0ed48d86758fdd3bee8bacc14118ed13fc316bae1dc839595a03b7234125925 I0401 11:31:06.679568 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-95bdf696d-qxkl2/registry_current.log with fingerprint=a14e4ec99cecffa66f7a15b6baddba21d0ae368ca0a5350dea623ea64c93b7ae I0401 11:31:06.679575 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-ccqzp/node-ca_current.log with fingerprint=90aeac473cadc13223f09ee5e21b69b1145f41c2f316c889e26052880832f658 I0401 11:31:06.679580 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-gmxhs/node-ca_current.log with fingerprint=e5a85fc1f0e64e4e5f369f18e91163ed7eabc4c624bc553fa5bde3505b63bee2 I0401 11:31:06.679596 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-7656455898-rmxff/router_current.log with fingerprint=c6ea92f6dafc11b641c4cbfe0ee02bd859dfa9d58abdc96bf45cff49c17ac07c I0401 11:31:06.679611 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-79675977c5-ss5g4/router_current.log with fingerprint=53311c6e96e00ed0168a1ea62bc64a3273d35f5174282bdfbe09051b96ef3492 I0401 11:31:06.679615 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-sd4lc/serve-healthcheck-canary_current.log with fingerprint=e8b93a89cf23be97d17ac428cb6591954ae95103441bc117c616c8376197ff02 I0401 11:31:06.679621 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.048301519s to process 24 records I0401 11:31:13.470172 1 tasks_processing.go:74] worker 43 stopped. I0401 11:31:13.470212 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0401 11:31:13.470224 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.839546134s to process 1 records I0401 11:31:14.237626 1 tasks_processing.go:74] worker 8 stopped. I0401 11:31:14.237902 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=1a12195152df53baccddfc9d0b61ab216e3b4b53237b871699c391e051196073 I0401 11:31:14.237918 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.606550071s to process 1 records E0401 11:31:14.237984 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.609s with: function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"support_secret\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json\"" I0401 11:31:14.239114 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "support_secret" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-vc9cf.json" I0401 11:31:14.239128 1 periodic.go:209] Running workloads gatherer I0401 11:31:14.239142 1 tasks_processing.go:45] number of workers: 2 I0401 11:31:14.239147 1 tasks_processing.go:69] worker 1 listening for tasks. I0401 11:31:14.239151 1 tasks_processing.go:71] worker 1 working on workload_info task. I0401 11:31:14.239157 1 tasks_processing.go:69] worker 0 listening for tasks. I0401 11:31:14.239171 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0401 11:31:14.245284 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-170.ec2.internal... I0401 11:31:14.266074 1 tasks_processing.go:74] worker 0 stopped. I0401 11:31:14.266094 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 26.888419ms to process 0 records I0401 11:31:18.042528 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0401 11:31:18.088543 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 21s for image data I0401 11:31:18.099525 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (12ms) I0401 11:31:18.110134 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (11ms) I0401 11:31:18.119829 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0401 11:31:18.130590 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (11ms) I0401 11:31:18.139986 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (9ms) I0401 11:31:18.149358 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (9ms) I0401 11:31:18.158709 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (9ms) I0401 11:31:18.169854 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (11ms) I0401 11:31:18.179504 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (10ms) I0401 11:31:18.188848 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (9ms) I0401 11:31:18.198598 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (10ms) I0401 11:31:18.297174 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (99ms) I0401 11:31:18.397827 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0401 11:31:18.497767 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0401 11:31:18.598153 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (100ms) I0401 11:31:18.697515 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (99ms) I0401 11:31:18.797490 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0401 11:31:18.897399 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0401 11:31:18.997590 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0401 11:31:19.098261 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (101ms) I0401 11:31:19.197687 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0401 11:31:19.297426 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0401 11:31:19.397574 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0401 11:31:19.498221 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0401 11:31:19.597312 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0401 11:31:19.700337 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (103ms) I0401 11:31:19.797384 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (97ms) I0401 11:31:19.897405 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0401 11:31:19.997257 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0401 11:31:20.097691 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0401 11:31:20.197273 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (100ms) I0401 11:31:20.297964 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (101ms) I0401 11:31:20.397213 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (99ms) I0401 11:31:20.496788 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0401 11:31:20.598497 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (102ms) I0401 11:31:20.696944 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (98ms) I0401 11:31:20.797443 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0401 11:31:20.897177 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0401 11:31:20.897212 1 tasks_processing.go:74] worker 1 stopped. I0401 11:31:20.897448 1 recorder.go:75] Recording config/workload_info with fingerprint=4eef73bcba57410985dcb6bbb6b29710f81f96fb3239a271f990cc1f4c9042f2 I0401 11:31:20.897465 1 gather.go:177] gatherer "workloads" function "workload_info" took 6.658053371s to process 1 records I0401 11:31:20.897492 1 periodic.go:256] Periodic gather workloads completed in 6.658s I0401 11:31:20.897504 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0401 11:31:20.897510 1 periodic.go:209] Running conditional gatherer I0401 11:31:20.902777 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0401 11:31:21.091523 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0401 11:31:21.099014 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0401 11:31:21.099101 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0401 11:31:21.104906 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0401 11:31:21.104929 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104938 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104943 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104948 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104953 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104957 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104962 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104966 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104971 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0401 11:31:21.104975 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0401 11:31:21.105013 1 tasks_processing.go:45] number of workers: 3 I0401 11:31:21.105026 1 tasks_processing.go:69] worker 2 listening for tasks. I0401 11:31:21.105032 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0401 11:31:21.105036 1 tasks_processing.go:69] worker 0 listening for tasks. I0401 11:31:21.105050 1 tasks_processing.go:69] worker 1 listening for tasks. I0401 11:31:21.105066 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0401 11:31:21.105077 1 tasks_processing.go:74] worker 0 stopped. I0401 11:31:21.105078 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0401 11:31:21.105148 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0401 11:31:21.105163 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.396µs to process 1 records I0401 11:31:21.105232 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0401 11:31:21.105241 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.929µs to process 1 records I0401 11:31:21.105251 1 tasks_processing.go:74] worker 1 stopped. I0401 11:31:21.105405 1 tasks_processing.go:74] worker 2 stopped. I0401 11:31:21.105421 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 365.301µs to process 0 records I0401 11:31:21.105439 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0401 11:31:21.105466 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0401 11:31:21.135977 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=3432b15339affb8b3923258e1905490794d977caade803c85cd5b94dc89f20ca I0401 11:31:21.136162 1 diskrecorder.go:70] Writing 125 records to /var/lib/insights-operator/insights-2026-04-01-113121.tar.gz I0401 11:31:21.145255 1 diskrecorder.go:51] Wrote 125 records to disk in 9ms I0401 11:31:21.145303 1 periodic.go:278] Gathering cluster info every 2h0m0s I0401 11:31:21.145322 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0401 11:32:30.628303 1 diskrecorder.go:223] Found files to send: insights-2026-04-01-113121.tar.gz I0401 11:32:30.628335 1 insightsuploader.go:150] Checking archives to upload periodically every 15m5.523975322s I0401 11:32:30.628343 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0401 11:32:30.640446 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0401 11:32:30.910526 1 requests.go:87] Successfully reported id=2026-04-01T11:32:30Z x-rh-insights-request-id=2a0fdf5ede034058b21f63b9940ba1d4, wrote=65036 I0401 11:32:30.910588 1 insightsuploader.go:187] Uploaded report successfully in 282.236435ms I0401 11:32:30.910609 1 controller.go:128] Initializing last reported time to 2026-04-01T11:32:30Z I0401 11:32:30.910653 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0401 11:32:30.910661 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0401 11:32:30.910668 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0401 11:32:30.917147 1 controller.go:512] The operator is healthy I0401 11:33:00.639429 1 controller.go:512] The operator is healthy I0401 11:33:32.260967 1 insightsreport.go:137] Pulling report from smart-proxy I0401 11:33:32.261010 1 insightsreport.go:149] Retrieving report I0401 11:33:32.267321 1 requests.go:110] Retrieving report for cluster: 027222a6-0fff-45b3-90d9-4d42ebdc4d30 I0401 11:33:32.267333 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/027222a6-0fff-45b3-90d9-4d42ebdc4d30/reports I0401 11:33:32.270119 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/027222a6-0fff-45b3-90d9-4d42ebdc4d30/reports I0401 11:33:32.513235 1 insightsreport.go:184] Report retrieved I0401 11:33:32.544068 1 insightsreport.go:239] Report retrieved correctly I0401 11:35:00.640137 1 controller.go:512] The operator is healthy I0401 11:36:00.621418 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 11:36:00.626771 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:36:00.626791 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 11:36:00.630641 1 secretconfigobserver.go:119] support secret does not exist I0401 11:37:00.640588 1 controller.go:512] The operator is healthy I0401 11:39:00.639500 1 controller.go:512] The operator is healthy I0401 11:41:00.621646 1 configmapobserver.go:84] configmaps "insights-config" not found I0401 11:41:00.631710 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 11:41:00.635936 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:41:00.635954 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 11:41:00.639039 1 controller.go:512] The operator is healthy I0401 11:41:00.639886 1 secretconfigobserver.go:119] support secret does not exist I0401 11:43:00.640902 1 controller.go:512] The operator is healthy I0401 11:45:00.640328 1 controller.go:512] The operator is healthy I0401 11:46:00.640576 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 11:46:00.645371 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:46:00.645390 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 11:46:00.649215 1 secretconfigobserver.go:119] support secret does not exist I0401 11:47:00.642956 1 controller.go:512] The operator is healthy I0401 11:47:41.435685 1 insightsuploader.go:146] Nothing to report since 2026-04-01T11:32:30Z I0401 11:49:00.638907 1 controller.go:512] The operator is healthy I0401 11:51:00.622573 1 configmapobserver.go:84] configmaps "insights-config" not found I0401 11:51:00.638461 1 controller.go:512] The operator is healthy I0401 11:51:00.650221 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 11:51:00.654319 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:51:00.654336 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 11:51:00.657762 1 secretconfigobserver.go:119] support secret does not exist I0401 11:53:00.640422 1 controller.go:512] The operator is healthy I0401 11:55:00.639683 1 controller.go:512] The operator is healthy I0401 11:56:00.658257 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 11:56:00.663830 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 11:56:00.663849 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 11:56:00.667657 1 secretconfigobserver.go:119] support secret does not exist I0401 11:57:00.638700 1 controller.go:512] The operator is healthy I0401 11:59:00.639984 1 controller.go:512] The operator is healthy I0401 12:01:00.623624 1 configmapobserver.go:84] configmaps "insights-config" not found I0401 12:01:00.627517 1 configmapobserver.go:84] configmaps "insights-config" not found I0401 12:01:00.638435 1 controller.go:512] The operator is healthy I0401 12:01:00.668427 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 12:01:00.672631 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 12:01:00.672648 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 12:01:00.676257 1 secretconfigobserver.go:119] support secret does not exist I0401 12:03:00.642059 1 controller.go:512] The operator is healthy I0401 12:03:49.372199 1 insightsuploader.go:146] Nothing to report since 2026-04-01T11:32:30Z I0401 12:05:00.641384 1 controller.go:512] The operator is healthy I0401 12:06:00.676626 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0401 12:06:00.681377 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0401 12:06:00.681397 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0401 12:06:00.684975 1 secretconfigobserver.go:119] support secret does not exist I0401 12:07:00.639548 1 controller.go:512] The operator is healthy