I0327 23:09:58.038050 1 cmd.go:253] Using service-serving-cert provided certificates I0327 23:09:58.038473 1 observer_polling.go:159] Starting file observer I0327 23:09:58.478226 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0327 23:09:58.478421 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0327 23:09:58.480103 1 secure_serving.go:57] Forcing use of http/1.1 only W0327 23:09:58.480127 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0327 23:09:58.480132 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0327 23:09:58.480137 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0327 23:09:58.480140 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0327 23:09:58.480143 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0327 23:09:58.480145 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0327 23:09:58.480166 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0327 23:09:58.483855 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0327 23:09:58.483909 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"e2f7ec00-e1f8-47ca-8ad6-a4694bfb1ed8", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0327 23:09:58.485105 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0327 23:09:58.485121 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0327 23:09:58.485114 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0327 23:09:58.485122 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0327 23:09:58.485134 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0327 23:09:58.485137 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0327 23:09:58.485385 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0327 23:09:58.485616 1 secure_serving.go:213] Serving securely on [::]:8443 I0327 23:09:58.485644 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0327 23:09:58.492370 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0327 23:09:58.492395 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0327 23:09:58.492495 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0327 23:09:58.496902 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:09:58.496920 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0327 23:09:58.501590 1 secretconfigobserver.go:119] support secret does not exist I0327 23:09:58.506245 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:09:58.510734 1 secretconfigobserver.go:119] support secret does not exist I0327 23:09:58.512493 1 recorder.go:161] Pruning old reports every 5h10m34s, max age is 288h0m0s I0327 23:09:58.517614 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0327 23:09:58.517621 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0327 23:09:58.517631 1 insightsreport.go:296] Starting report retriever I0327 23:09:58.517633 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0327 23:09:58.517637 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0327 23:09:58.517658 1 periodic.go:209] Running clusterconfig gatherer I0327 23:09:58.517708 1 tasks_processing.go:45] number of workers: 64 I0327 23:09:58.517735 1 tasks_processing.go:69] worker 2 listening for tasks. I0327 23:09:58.517740 1 tasks_processing.go:69] worker 0 listening for tasks. I0327 23:09:58.517749 1 tasks_processing.go:69] worker 25 listening for tasks. I0327 23:09:58.517749 1 tasks_processing.go:69] worker 1 listening for tasks. I0327 23:09:58.517755 1 tasks_processing.go:69] worker 14 listening for tasks. I0327 23:09:58.517751 1 tasks_processing.go:69] worker 39 listening for tasks. I0327 23:09:58.517773 1 tasks_processing.go:71] worker 39 working on machine_sets task. I0327 23:09:58.517770 1 tasks_processing.go:69] worker 13 listening for tasks. I0327 23:09:58.517779 1 tasks_processing.go:69] worker 15 listening for tasks. I0327 23:09:58.517778 1 tasks_processing.go:69] worker 20 listening for tasks. I0327 23:09:58.517779 1 tasks_processing.go:69] worker 40 listening for tasks. I0327 23:09:58.517780 1 tasks_processing.go:69] worker 19 listening for tasks. I0327 23:09:58.517772 1 tasks_processing.go:69] worker 16 listening for tasks. I0327 23:09:58.517785 1 tasks_processing.go:69] worker 41 listening for tasks. I0327 23:09:58.517795 1 tasks_processing.go:69] worker 23 listening for tasks. I0327 23:09:58.517784 1 tasks_processing.go:69] worker 21 listening for tasks. I0327 23:09:58.517798 1 tasks_processing.go:69] worker 5 listening for tasks. I0327 23:09:58.517800 1 tasks_processing.go:69] worker 18 listening for tasks. I0327 23:09:58.517797 1 tasks_processing.go:69] worker 17 listening for tasks. I0327 23:09:58.517787 1 tasks_processing.go:69] worker 3 listening for tasks. I0327 23:09:58.517810 1 tasks_processing.go:69] worker 11 listening for tasks. I0327 23:09:58.517790 1 tasks_processing.go:69] worker 22 listening for tasks. I0327 23:09:58.517815 1 tasks_processing.go:69] worker 8 listening for tasks. I0327 23:09:58.517814 1 tasks_processing.go:69] worker 7 listening for tasks. I0327 23:09:58.517821 1 tasks_processing.go:69] worker 58 listening for tasks. I0327 23:09:58.517816 1 tasks_processing.go:69] worker 24 listening for tasks. I0327 23:09:58.517828 1 tasks_processing.go:69] worker 63 listening for tasks. I0327 23:09:58.517826 1 tasks_processing.go:69] worker 47 listening for tasks. I0327 23:09:58.517829 1 tasks_processing.go:69] worker 62 listening for tasks. I0327 23:09:58.517833 1 tasks_processing.go:69] worker 52 listening for tasks. I0327 23:09:58.517835 1 tasks_processing.go:71] worker 0 working on machines task. I0327 23:09:58.517836 1 tasks_processing.go:69] worker 53 listening for tasks. I0327 23:09:58.517841 1 tasks_processing.go:69] worker 49 listening for tasks. I0327 23:09:58.517838 1 tasks_processing.go:69] worker 50 listening for tasks. I0327 23:09:58.517842 1 tasks_processing.go:69] worker 54 listening for tasks. I0327 23:09:58.517849 1 tasks_processing.go:69] worker 37 listening for tasks. I0327 23:09:58.517843 1 tasks_processing.go:69] worker 59 listening for tasks. I0327 23:09:58.517756 1 tasks_processing.go:71] worker 1 working on proxies task. I0327 23:09:58.517850 1 tasks_processing.go:69] worker 61 listening for tasks. I0327 23:09:58.517859 1 tasks_processing.go:71] worker 14 working on service_accounts task. I0327 23:09:58.517862 1 tasks_processing.go:71] worker 16 working on metrics task. I0327 23:09:58.517856 1 tasks_processing.go:69] worker 34 listening for tasks. I0327 23:09:58.517867 1 tasks_processing.go:71] worker 41 working on nodes task. I0327 23:09:58.517866 1 tasks_processing.go:71] worker 63 working on storage_cluster task. I0327 23:09:58.517872 1 tasks_processing.go:69] worker 36 listening for tasks. I0327 23:09:58.517866 1 tasks_processing.go:69] worker 35 listening for tasks. I0327 23:09:58.517877 1 tasks_processing.go:71] worker 21 working on pdbs task. I0327 23:09:58.517880 1 tasks_processing.go:71] worker 50 working on overlapping_namespace_uids task. I0327 23:09:58.517883 1 tasks_processing.go:71] worker 35 working on openshift_machine_api_events task. I0327 23:09:58.517883 1 tasks_processing.go:71] worker 54 working on certificate_signing_requests task. I0327 23:09:58.517880 1 tasks_processing.go:71] worker 8 working on sap_pods task. I0327 23:09:58.517934 1 tasks_processing.go:71] worker 24 working on openstack_dataplanedeployments task. I0327 23:09:58.517946 1 tasks_processing.go:71] worker 37 working on authentication task. I0327 23:09:58.517801 1 tasks_processing.go:69] worker 44 listening for tasks. I0327 23:09:58.517972 1 tasks_processing.go:71] worker 11 working on nodenetworkconfigurationpolicies task. I0327 23:09:58.517975 1 tasks_processing.go:71] worker 44 working on storage_classes task. I0327 23:09:58.517976 1 tasks_processing.go:71] worker 22 working on monitoring_persistent_volumes task. I0327 23:09:58.517980 1 tasks_processing.go:71] worker 49 working on machine_config_pools task. I0327 23:09:58.517984 1 tasks_processing.go:71] worker 20 working on olm_operators task. I0327 23:09:58.517988 1 tasks_processing.go:71] worker 40 working on cluster_apiserver task. I0327 23:09:58.517993 1 tasks_processing.go:71] worker 5 working on machine_healthchecks task. I0327 23:09:58.517872 1 tasks_processing.go:71] worker 34 working on silenced_alerts task. I0327 23:09:58.518017 1 tasks_processing.go:71] worker 7 working on machine_autoscalers task. I0327 23:09:58.517756 1 tasks_processing.go:71] worker 25 working on jaegers task. I0327 23:09:58.518026 1 tasks_processing.go:71] worker 3 working on openstack_version task. I0327 23:09:58.517871 1 tasks_processing.go:71] worker 62 working on support_secret task. I0327 23:09:58.518027 1 tasks_processing.go:71] worker 19 working on config_maps task. I0327 23:09:58.518079 1 tasks_processing.go:71] worker 13 working on mutating_webhook_configurations task. I0327 23:09:58.518080 1 tasks_processing.go:71] worker 15 working on crds task. I0327 23:09:58.517880 1 tasks_processing.go:71] worker 53 working on feature_gates task. I0327 23:09:58.517989 1 tasks_processing.go:71] worker 18 working on nodenetworkstates task. I0327 23:09:58.518152 1 tasks_processing.go:71] worker 58 working on active_alerts task. I0327 23:09:58.517806 1 tasks_processing.go:69] worker 10 listening for tasks. I0327 23:09:58.518258 1 tasks_processing.go:71] worker 10 working on node_logs task. I0327 23:09:58.517810 1 tasks_processing.go:69] worker 6 listening for tasks. I0327 23:09:58.517811 1 tasks_processing.go:69] worker 56 listening for tasks. I0327 23:09:58.517790 1 tasks_processing.go:69] worker 42 listening for tasks. I0327 23:09:58.517816 1 tasks_processing.go:69] worker 57 listening for tasks. I0327 23:09:58.517818 1 tasks_processing.go:69] worker 46 listening for tasks. I0327 23:09:58.517822 1 tasks_processing.go:69] worker 48 listening for tasks. I0327 23:09:58.517823 1 tasks_processing.go:69] worker 33 listening for tasks. I0327 23:09:58.517825 1 tasks_processing.go:69] worker 51 listening for tasks. I0327 23:09:58.517820 1 tasks_processing.go:69] worker 12 listening for tasks. I0327 23:09:58.517828 1 tasks_processing.go:69] worker 26 listening for tasks. I0327 23:09:58.517832 1 tasks_processing.go:71] worker 2 working on machine_configs task. I0327 23:09:58.517802 1 tasks_processing.go:69] worker 9 listening for tasks. I0327 23:09:58.518437 1 tasks_processing.go:71] worker 48 working on networks task. I0327 23:09:58.518443 1 tasks_processing.go:71] worker 9 working on openstack_dataplanenodesets task. I0327 23:09:58.518448 1 tasks_processing.go:71] worker 42 working on image_pruners task. I0327 23:09:58.518440 1 tasks_processing.go:71] worker 6 working on ingress_certificates task. I0327 23:09:58.518518 1 tasks_processing.go:71] worker 57 working on ingress task. I0327 23:09:58.517834 1 tasks_processing.go:69] worker 27 listening for tasks. I0327 23:09:58.517839 1 tasks_processing.go:69] worker 28 listening for tasks. I0327 23:09:58.517843 1 tasks_processing.go:69] worker 30 listening for tasks. I0327 23:09:58.517843 1 tasks_processing.go:69] worker 31 listening for tasks. I0327 23:09:58.517849 1 tasks_processing.go:69] worker 32 listening for tasks. I0327 23:09:58.517850 1 tasks_processing.go:69] worker 29 listening for tasks. I0327 23:09:58.517855 1 tasks_processing.go:69] worker 38 listening for tasks. I0327 23:09:58.517862 1 tasks_processing.go:71] worker 17 working on operators_pods_and_events task. I0327 23:09:58.517793 1 tasks_processing.go:69] worker 4 listening for tasks. I0327 23:09:58.517864 1 tasks_processing.go:71] worker 61 working on validating_webhook_configurations task. I0327 23:09:58.518684 1 tasks_processing.go:71] worker 4 working on openshift_logging task. I0327 23:09:58.518697 1 tasks_processing.go:71] worker 51 working on sap_config task. I0327 23:09:58.518704 1 tasks_processing.go:71] worker 12 working on sap_datahubs task. I0327 23:09:58.518444 1 tasks_processing.go:71] worker 56 working on pod_network_connectivity_checks task. I0327 23:09:58.518852 1 tasks_processing.go:71] worker 27 working on operators task. I0327 23:09:58.517834 1 tasks_processing.go:69] worker 60 listening for tasks. I0327 23:09:58.518978 1 tasks_processing.go:71] worker 60 working on dvo_metrics task. I0327 23:09:58.519020 1 tasks_processing.go:71] worker 31 working on aggregated_monitoring_cr_names task. I0327 23:09:58.519028 1 tasks_processing.go:71] worker 29 working on install_plans task. I0327 23:09:58.517863 1 tasks_processing.go:71] worker 59 working on qemu_kubevirt_launcher_logs task. I0327 23:09:58.519047 1 tasks_processing.go:71] worker 33 working on openstack_controlplanes task. I0327 23:09:58.519157 1 tasks_processing.go:71] worker 32 working on container_runtime_configs task. I0327 23:09:58.517875 1 tasks_processing.go:71] worker 52 working on version task. I0327 23:09:58.517867 1 tasks_processing.go:71] worker 47 working on container_images task. I0327 23:09:58.517978 1 tasks_processing.go:71] worker 23 working on image task. I0327 23:09:58.517810 1 tasks_processing.go:69] worker 55 listening for tasks. I0327 23:09:58.519502 1 tasks_processing.go:71] worker 55 working on ceph_cluster task. I0327 23:09:58.517796 1 tasks_processing.go:69] worker 43 listening for tasks. I0327 23:09:58.518431 1 tasks_processing.go:71] worker 26 working on cost_management_metrics_configs task. I0327 23:09:58.519632 1 tasks_processing.go:71] worker 43 working on tsdb_status task. I0327 23:09:58.519182 1 tasks_processing.go:71] worker 28 working on image_registries task. I0327 23:09:58.518433 1 tasks_processing.go:71] worker 46 working on clusterroles task. I0327 23:09:58.517877 1 tasks_processing.go:71] worker 36 working on lokistack task. I0327 23:09:58.519187 1 tasks_processing.go:71] worker 30 working on infrastructures task. I0327 23:09:58.519192 1 tasks_processing.go:71] worker 38 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0327 23:09:58.517805 1 tasks_processing.go:69] worker 45 listening for tasks. I0327 23:09:58.519940 1 tasks_processing.go:71] worker 45 working on schedulers task. I0327 23:09:58.520684 1 tasks_processing.go:71] worker 39 working on oauths task. I0327 23:09:58.520728 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 2.864551ms to process 0 records I0327 23:09:58.521810 1 tasks_processing.go:74] worker 18 stopped. I0327 23:09:58.521837 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 3.667065ms to process 0 records I0327 23:09:58.522386 1 tasks_processing.go:74] worker 0 stopped. E0327 23:09:58.522401 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0327 23:09:58.522409 1 gather.go:177] gatherer "clusterconfig" function "machines" took 4.540499ms to process 0 records E0327 23:09:58.524216 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.43.21:9091: connect: connection refused I0327 23:09:58.524240 1 tasks_processing.go:74] worker 43 stopped. E0327 23:09:58.524249 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.43.21:9091: connect: connection refused I0327 23:09:58.524264 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.565342ms to process 0 records E0327 23:09:58.524261 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.43.21:9091: connect: connection refused I0327 23:09:58.524278 1 tasks_processing.go:74] worker 16 stopped. E0327 23:09:58.524285 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.43.21:9091: connect: connection refused I0327 23:09:58.524297 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.409158ms to process 0 records I0327 23:09:58.524405 1 tasks_processing.go:74] worker 25 stopped. I0327 23:09:58.524426 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 6.376303ms to process 0 records I0327 23:09:58.524438 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 6.399029ms to process 0 records I0327 23:09:58.524453 1 tasks_processing.go:74] worker 7 stopped. I0327 23:09:58.527398 1 tasks_processing.go:74] worker 9 stopped. I0327 23:09:58.527408 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 8.945984ms to process 0 records I0327 23:09:58.527416 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 9.526414ms to process 0 records I0327 23:09:58.527421 1 tasks_processing.go:74] worker 63 stopped. I0327 23:09:58.527541 1 tasks_processing.go:74] worker 12 stopped. I0327 23:09:58.527554 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 8.821375ms to process 0 records I0327 23:09:58.527615 1 tasks_processing.go:74] worker 49 stopped. I0327 23:09:58.527627 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 9.619407ms to process 0 records I0327 23:09:58.527715 1 recorder.go:75] Recording config/proxy with fingerprint=1463707542304912424744e89d630571c5d959cb689eab50eaf6fcc0bc41acf5 I0327 23:09:58.527726 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 9.75123ms to process 1 records I0327 23:09:58.527732 1 tasks_processing.go:74] worker 1 stopped. I0327 23:09:58.527742 1 tasks_processing.go:74] worker 37 stopped. I0327 23:09:58.527975 1 recorder.go:75] Recording config/authentication with fingerprint=34f45bd9d724a554946fc0117cf0340330c2c93819fcfe5fc86332d109dc0c5c I0327 23:09:58.527989 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 9.780596ms to process 1 records I0327 23:09:58.528068 1 recorder.go:75] Recording config/featuregate with fingerprint=3fda1d00d9ed28b317d16a61798db1cf5b86f4ab86e1ebaa3c58d9a9f9c8805c I0327 23:09:58.528072 1 tasks_processing.go:74] worker 53 stopped. I0327 23:09:58.528077 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 9.648833ms to process 1 records I0327 23:09:58.530449 1 tasks_processing.go:74] worker 3 stopped. I0327 23:09:58.530457 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 12.411315ms to process 0 records E0327 23:09:58.530465 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0327 23:09:58.530473 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 12.455229ms to process 0 records I0327 23:09:58.530480 1 tasks_processing.go:74] worker 5 stopped. I0327 23:09:58.535304 1 tasks_processing.go:74] worker 35 stopped. I0327 23:09:58.535318 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 17.415429ms to process 0 records I0327 23:09:58.535331 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 16.616947ms to process 0 records I0327 23:09:58.535337 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 17.358456ms to process 0 records I0327 23:09:58.535348 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 16.252405ms to process 0 records I0327 23:09:58.535359 1 tasks_processing.go:74] worker 33 stopped. I0327 23:09:58.535364 1 tasks_processing.go:74] worker 24 stopped. I0327 23:09:58.535367 1 tasks_processing.go:74] worker 4 stopped. I0327 23:09:58.543063 1 tasks_processing.go:74] worker 11 stopped. I0327 23:09:58.543080 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 25.08084ms to process 0 records I0327 23:09:58.543113 1 tasks_processing.go:74] worker 62 stopped. E0327 23:09:58.543124 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0327 23:09:58.543133 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 25.073445ms to process 0 records I0327 23:09:58.543142 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 24.831236ms to process 0 records I0327 23:09:58.543147 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 24.424293ms to process 0 records I0327 23:09:58.543152 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 25.225404ms to process 0 records I0327 23:09:58.543159 1 tasks_processing.go:74] worker 8 stopped. I0327 23:09:58.543166 1 tasks_processing.go:74] worker 10 stopped. I0327 23:09:58.543171 1 tasks_processing.go:74] worker 51 stopped. I0327 23:09:58.543187 1 tasks_processing.go:74] worker 36 stopped. I0327 23:09:58.543195 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 23.446496ms to process 0 records I0327 23:09:58.543201 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 23.97196ms to process 0 records I0327 23:09:58.543206 1 tasks_processing.go:74] worker 32 stopped. I0327 23:09:58.543268 1 tasks_processing.go:74] worker 55 stopped. I0327 23:09:58.543280 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 23.759727ms to process 0 records I0327 23:09:58.543288 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 23.715145ms to process 0 records I0327 23:09:58.543294 1 tasks_processing.go:74] worker 26 stopped. E0327 23:09:58.543296 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0327 23:09:58.543304 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 24.458777ms to process 0 records I0327 23:09:58.543309 1 tasks_processing.go:74] worker 56 stopped. I0327 23:09:58.543354 1 tasks_processing.go:74] worker 13 stopped. I0327 23:09:58.543548 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=c8a6f5dfcf50fd41443223cc996f67ab20746aaee3d9da939a542afdf13ff8f5 I0327 23:09:58.543580 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=ad26c18d6dcd2b8d903247d036bd49da353311f5b988487a812527f5b9a366c7 I0327 23:09:58.543605 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=471e99a6bc56028f54389680f9791e12d6b45d0e611fd3db01d0ebe6c1015e3d I0327 23:09:58.543613 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 25.258985ms to process 3 records I0327 23:09:58.543690 1 tasks_processing.go:74] worker 40 stopped. I0327 23:09:58.543725 1 recorder.go:75] Recording config/apiserver with fingerprint=f4e14d41a0e4c0f374406f6871a80a7399f35c6bc1f331779f425202f8b3e0b6 I0327 23:09:58.543732 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 25.354462ms to process 1 records I0327 23:09:58.543808 1 tasks_processing.go:74] worker 42 stopped. I0327 23:09:58.543939 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=b4bffd5b7e1a54cade3eca4a5f43d3f676770d0f4524c2d7e92d771cfa710c15 I0327 23:09:58.543950 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 24.940482ms to process 1 records I0327 23:09:58.544026 1 tasks_processing.go:74] worker 48 stopped. I0327 23:09:58.544046 1 recorder.go:75] Recording config/network with fingerprint=20c9eae8a42d5992d9526e9fe4b064b5ddff22641c93288ed231c14c5d1ad8e5 I0327 23:09:58.544056 1 gather.go:177] gatherer "clusterconfig" function "networks" took 24.981533ms to process 1 records I0327 23:09:58.544929 1 tasks_processing.go:74] worker 22 stopped. I0327 23:09:58.544947 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 26.940782ms to process 0 records I0327 23:09:58.545058 1 tasks_processing.go:74] worker 21 stopped. I0327 23:09:58.545180 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=7bb46c5efe9dfbbd72e7c2e9551463dbd36cf8453766e5aefe023ed34ae64a16 I0327 23:09:58.545206 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=e3564487673ffe02ed390dce7dedf4698dffa3bd893821f84b106cecce1ef1ed I0327 23:09:58.545226 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=238a5481e374df1820b0970abef443dba14214332f7e0b100a941d26c7cee470 I0327 23:09:58.545249 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=0bb9a072b0e589cebb1535f718639eb0516dfbc6cd1138d20b6698505da43ed1 I0327 23:09:58.545276 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=a670d077b6f746b1b06122d9059996f0f2ee7ed5c7213063078397f219b14faf I0327 23:09:58.545299 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=e9a8ff48611f02482442d12bd1a3b44fd5e2ef15c0686479e5b0a091bda9e836 I0327 23:09:58.545326 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=5d655ff9e37f1ce75a8c12650e791da5ed1f31dfb455b86e9850ea19cf42fc68 I0327 23:09:58.545354 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=72d3584123c6504241273ab4365f4a764066d4840b0c76f4a708859ed3a00485 I0327 23:09:58.545371 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0327 23:09:58.545377 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=5bf5c22b375e3ab9ef9780d5b655f7398babba49dcf5968ab5f4318ba68331d2 I0327 23:09:58.545386 1 controller.go:326] The initial operator extension status is healthy I0327 23:09:58.545397 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0327 23:09:58.545400 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=25a41eaee33c02f9b566be2b24acd623700411f77af07e6aee25c8a143a6a6fa I0327 23:09:58.545403 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0327 23:09:58.545408 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0327 23:09:58.545424 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=0a79938e320529a1521f5a22a24ca2aceab189cef1bf99aed41f658e84c46274 I0327 23:09:58.545429 1 controller.go:489] The operator is still being initialized I0327 23:09:58.545433 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 27.170853ms to process 11 records I0327 23:09:58.545437 1 controller.go:512] The operator is healthy I0327 23:09:58.545538 1 tasks_processing.go:74] worker 57 stopped. I0327 23:09:58.545567 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0327 23:09:58.545572 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0327 23:09:58.545587 1 recorder.go:75] Recording config/ingress with fingerprint=bb668289896a6ca221b8996be8e3cac82f348a6df6b51e4af77276e9b6df07d4 I0327 23:09:58.545597 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 26.58697ms to process 1 records I0327 23:09:58.545636 1 tasks_processing.go:74] worker 50 stopped. I0327 23:09:58.545669 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0327 23:09:58.545685 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 27.746ms to process 1 records W0327 23:09:58.545687 1 operator.go:288] started I0327 23:09:58.545706 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0327 23:09:58.546800 1 tasks_processing.go:74] worker 23 stopped. I0327 23:09:58.546981 1 recorder.go:75] Recording config/image with fingerprint=a688f8d4095cdd6b5066e2a82a683dd2142c6542a88fa3ba3967b3a5359e37f5 I0327 23:09:58.547055 1 gather.go:177] gatherer "clusterconfig" function "image" took 27.311419ms to process 1 records I0327 23:09:58.547350 1 tasks_processing.go:74] worker 30 stopped. I0327 23:09:58.548971 1 recorder.go:75] Recording config/infrastructure with fingerprint=44e5bbe650214d686759fd2a90a8c805956d2d3e91fa5419199e7e78f59f873d I0327 23:09:58.548995 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 27.005193ms to process 1 records I0327 23:09:58.549042 1 tasks_processing.go:74] worker 44 stopped. I0327 23:09:58.549089 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=ada18968974f08ab8dbca120fe911f9c12a2dc052242da60e646a8c87e0ffd3a I0327 23:09:58.549110 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=4ecfbe571dcf9e7462813f0bc853d59ac6994034f6e10e352b12899d3f7966ea I0327 23:09:58.549122 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 28.857054ms to process 2 records I0327 23:09:58.549192 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=c4e765e78d0f63581fcb7233b2c3c11292f92dad518f5038ced1308bbb2207e0 I0327 23:09:58.549202 1 tasks_processing.go:74] worker 45 stopped. I0327 23:09:58.549205 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 27.16417ms to process 1 records I0327 23:09:58.549282 1 tasks_processing.go:74] worker 28 stopped. W0327 23:09:58.549427 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 23:09:58.549683 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=dd1b27401c2e21342fb7bfeea1dc0de8a8afd7d586db2437a6f7b89c771c5f7c I0327 23:09:58.549699 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 28.955958ms to process 1 records I0327 23:09:58.550345 1 tasks_processing.go:74] worker 54 stopped. I0327 23:09:58.550359 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 32.427276ms to process 0 records I0327 23:09:58.550751 1 gather_logs.go:145] no pods in namespace were found I0327 23:09:58.550764 1 tasks_processing.go:74] worker 59 stopped. I0327 23:09:58.550771 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 31.724891ms to process 0 records I0327 23:09:58.551179 1 tasks_processing.go:74] worker 39 stopped. I0327 23:09:58.551472 1 recorder.go:75] Recording config/oauth with fingerprint=47558a5dc62b6f6c8c776c5dbe97a215cf20b2a9ef7bacd2550074b5900967f4 I0327 23:09:58.551498 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 30.435572ms to process 1 records I0327 23:09:58.552108 1 tasks_processing.go:74] worker 17 stopped. I0327 23:09:58.552122 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 33.534852ms to process 0 records I0327 23:09:58.559158 1 tasks_processing.go:74] worker 15 stopped. I0327 23:09:58.559636 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=6c7504b2206e28107f9ea2795a148bf4ee01c62a265e073f5656fb5a36c33456 I0327 23:09:58.559857 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=0a3cb41eab329dc7c436cad059cdc023a9ac0ab3df269a84cda8623506aba836 I0327 23:09:58.559868 1 gather.go:177] gatherer "clusterconfig" function "crds" took 41.061695ms to process 2 records I0327 23:09:58.559917 1 tasks_processing.go:74] worker 20 stopped. I0327 23:09:58.560231 1 recorder.go:75] Recording config/olm_operators with fingerprint=6ea90e3f6dc5ed1df9571c12ade228fe2e4057e279972fc038117cfe432c13e0 I0327 23:09:58.560261 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 41.840624ms to process 1 records I0327 23:09:58.562268 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0327 23:09:58.562287 1 controller.go:212] Source scaController *sca.Controller is not ready I0327 23:09:58.562293 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0327 23:09:58.562298 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0327 23:09:58.562302 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0327 23:09:58.562327 1 controller.go:489] The operator is still being initialized I0327 23:09:58.562336 1 controller.go:512] The operator is healthy I0327 23:09:58.563474 1 tasks_processing.go:74] worker 46 stopped. I0327 23:09:58.563648 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=06e8ceb1fb33ab1c529530d92298291429fe2712ea922ca4f2d642155747d718 I0327 23:09:58.563729 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=09cbb2e4b635122e3f3a8ac2f169e6a405204028a4e74ace9a1ba272a3895b72 I0327 23:09:58.563738 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 43.762492ms to process 2 records I0327 23:09:58.571494 1 tasks_processing.go:74] worker 31 stopped. I0327 23:09:58.571517 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 52.452319ms to process 0 records I0327 23:09:58.578511 1 tasks_processing.go:74] worker 47 stopped. I0327 23:09:58.579825 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-lrg4j with fingerprint=5f3f60136bb223deaffe0a31830b35a2fa7288ceb84fe27d4c8b0453efe5fab8 I0327 23:09:58.579945 1 recorder.go:75] Recording config/running_containers with fingerprint=c83d9bf39bab714eb690c1d25062eb3b50e611c7200bc48feb78015d861358e9 I0327 23:09:58.579956 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 59.058242ms to process 2 records I0327 23:09:58.579969 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 58.616914ms to process 0 records I0327 23:09:58.579978 1 tasks_processing.go:74] worker 38 stopped. I0327 23:09:58.581648 1 tasks_processing.go:74] worker 6 stopped. E0327 23:09:58.581663 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0327 23:09:58.581669 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pabktg44jfgggbdbhfgln3ik84drlf2-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pabktg44jfgggbdbhfgln3ik84drlf2-primary-cert-bundle-secret" not found I0327 23:09:58.581725 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=32592155288f01b8b3f6c9e16176ec3f42c31edd1c8479c7bae392d1dcd13dd3 I0327 23:09:58.581738 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 63.146841ms to process 1 records I0327 23:09:58.585346 1 tasks_processing.go:74] worker 19 stopped. I0327 23:09:58.585369 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0327 23:09:58.585370 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file E0327 23:09:58.585371 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0327 23:09:58.585386 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0327 23:09:58.585391 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0327 23:09:58.585405 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0327 23:09:58.585411 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0327 23:09:58.585435 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0327 23:09:58.585444 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0327 23:09:58.585448 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0327 23:09:58.585458 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0327 23:09:58.585521 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0327 23:09:58.585531 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0327 23:09:58.585537 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 67.297898ms to process 7 records I0327 23:09:58.592785 1 base_controller.go:82] Caches are synced for ConfigController I0327 23:09:58.592798 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0327 23:09:58.595473 1 tasks_processing.go:74] worker 61 stopped. I0327 23:09:58.595581 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=a407db8bdc778621e71442f2041d21de20ffcbaa57694e05f37ee795d8ecd3ea I0327 23:09:58.595648 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=af01315cfdbe182a4c5741ef9362de1ff9c29baeb5650a7710421e48dd99eada I0327 23:09:58.595737 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=95e12ef6c0e6725c37edd10be40767fa535c359db22b1ef82d5178488d715c24 I0327 23:09:58.595775 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=285d5fc2201ea14dde9f36a83627d2618a4c25967048c1e333568dd4f56aa881 I0327 23:09:58.595809 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=d5c86b7467adafc4f3f922e8b20d6e8d19632f1e8b44da9720608612218056ba I0327 23:09:58.595844 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=b26efd0046a6c26523fda3dfe79a52b16a085e2f0855ed1cba7dff701add3243 I0327 23:09:58.595877 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=9b4e47c878450a088dfadaa19e09afc264793873f52001b0542c18edc4efe529 I0327 23:09:58.595931 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=c9ec666341148f617ccc06fda08dbe3daa1fddbc4b20df6fa4707a9cb57528e5 I0327 23:09:58.595965 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=93b16903cea9b92237f027b2b140ad141fcba38395e082141ca776aa868ccbf8 I0327 23:09:58.596013 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=c83f1024a3e7ffbb5898dca96899710929ef3803b431b1f4a3bc09857e195cc3 I0327 23:09:58.596046 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=7dcc4d66f8a69697aca3bb01cb77a11ea3a4acf27af941a106d540f68e9ec976 I0327 23:09:58.596079 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=434c82610bcc58bf62e5e72ff57846a282b12d61b82108bd155fe076292e30c9 I0327 23:09:58.596112 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=4d6dc058ca3b970351aa54a6422db1675adbab5f1e4fe01b15c7a1987c937f71 I0327 23:09:58.596142 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 76.798434ms to process 13 records I0327 23:09:58.598343 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:09:58.637463 1 tasks_processing.go:74] worker 41 stopped. I0327 23:09:58.637792 1 recorder.go:75] Recording config/node/ip-10-0-0-196.ec2.internal with fingerprint=cb8f8fa61209d945611fc69b3d9a329063b243592a2f3b30288efe8c6d83e05e I0327 23:09:58.637856 1 recorder.go:75] Recording config/node/ip-10-0-1-41.ec2.internal with fingerprint=9ab5906f409031a5a42268510b3586c47f3977a08f399ef620aabf39f9345102 I0327 23:09:58.637923 1 recorder.go:75] Recording config/node/ip-10-0-2-65.ec2.internal with fingerprint=b7c474bc1efedee206571cd755e1e913a85343cbae21c4e743a0b11a7561d194 I0327 23:09:58.637933 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 119.580388ms to process 3 records I0327 23:09:58.639299 1 tasks_processing.go:74] worker 52 stopped. I0327 23:09:58.639523 1 recorder.go:75] Recording config/version with fingerprint=5e325b636c50d9a6a20f6a0a5d2bd5354b5d0091a64c96230f8488400d00c563 I0327 23:09:58.639534 1 recorder.go:75] Recording config/id with fingerprint=b3f9d11dedcfbc9dfa9fa95851d8c110a48303a4bec327e2d96a3285f9448126 I0327 23:09:58.639541 1 gather.go:177] gatherer "clusterconfig" function "version" took 120.010738ms to process 2 records I0327 23:09:58.639921 1 tasks_processing.go:74] worker 2 stopped. I0327 23:09:58.639943 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0327 23:09:58.639956 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 121.495235ms to process 1 records I0327 23:09:58.645534 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0327 23:09:58.645828 1 base_controller.go:82] Caches are synced for LoggingSyncer I0327 23:09:58.645842 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0327 23:09:58.656241 1 cluster_transfer.go:102] no available accepted cluster transfer I0327 23:09:58.656256 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0327 23:09:58.847695 1 sca.go:266] etc-pki-entitlement secret successfully created I0327 23:09:58.847717 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated W0327 23:09:59.548827 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 23:09:59.578695 1 tasks_processing.go:74] worker 34 stopped. I0327 23:09:59.578717 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0327 23:09:59.578728 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.060669743s to process 1 records I0327 23:09:59.578746 1 tasks_processing.go:74] worker 58 stopped. I0327 23:09:59.578814 1 recorder.go:75] Recording config/alerts with fingerprint=c17c035986913a629d9894c7e043efa987ecfe2e8cdca19b660b1f8bbec47bab I0327 23:09:59.578829 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.060566379s to process 1 records W0327 23:10:00.548379 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 23:10:00.984966 1 tasks_processing.go:74] worker 27 stopped. I0327 23:10:00.985081 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=83963122b505007489ef8ca5cbc397c242b0dbebac6ed1cf8db2474ddad3f13a I0327 23:10:00.985128 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0327 23:10:00.985155 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=116e7d49c161dfb67c431877f4d588748a8c6661be83facf17b4bfd19c6516b6 I0327 23:10:00.985166 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0327 23:10:00.985187 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=36f43fcf73d1b19ffd85a90f8d811c518a98d068d70c77a610ed8c30a2654101 I0327 23:10:00.985203 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0327 23:10:00.985222 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=a4613b62bebe1e28733ff3871e722899a08faa11d32213e4358ffcf00af8c5ab I0327 23:10:00.985248 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=af496e85ee5c6f804f199a9221a7ae099f9fce966722b30aa25b058c2f71f4b6 I0327 23:10:00.985271 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=0459493f8205cec0cf0d450a85a13e47e2ff4043b018f69155dd1f66e88f68a6 I0327 23:10:00.985302 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=e4a940ff744e9764e0fc8f300c136092ba2606a671db33cf5d0dc313162393b1 I0327 23:10:00.985573 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=43a27efa96c928ba8924038f5ae891a831a3567dc94efda51baa3466194f0219 I0327 23:10:00.985592 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=69ed090c13ecacf3b5439dd692b7627e935b19ca6b4e1b0054d8f02dd0ac8ab7 I0327 23:10:00.985601 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0327 23:10:00.985617 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=331010cd3810386e2b27d1dc95fe85e32f090ed5749df59305703e46a28648c2 I0327 23:10:00.985628 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0327 23:10:00.985643 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=41137b5ad79bdf00785f9bc65798dca053668f627b054f06c8e6ddea53fd895b I0327 23:10:00.985650 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0327 23:10:00.985670 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=a43d839f8f41b5a5b027921ad49574f2a72ff7a9cc0cd8d8a2ccb31a872084d8 I0327 23:10:00.985680 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0327 23:10:00.985699 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=56c8ba1f5fc9fc8b4c53b7f342ed2123e67e0e2fe6395ad3544484dd04e85325 I0327 23:10:00.985816 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=8f356e7b22e2722867b1d7f9b1bfda0d43aff56f48f56414bdb84d4580536a5b I0327 23:10:00.985827 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0327 23:10:00.985834 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0327 23:10:00.985855 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0327 23:10:00.985876 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=333375bfe7346ebb435cd7803b3c1ba1f0ee6435953807dec39e1404374ffbcd I0327 23:10:00.985918 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=69ea5e7b9ca0b8da3df171c70895820baedfeaa8e1b1acc924c1bc44aab58eb7 I0327 23:10:00.985928 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0327 23:10:00.985944 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=449d7a70fe56bba91e84dfadbce75e996d3ec456232bb1cd01a4b8d6ce602093 I0327 23:10:00.985955 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0327 23:10:00.985972 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=a14dde01cb271e5def1932848ae909ce8f7be8f4e71e75aed688b9cdb456c82d I0327 23:10:00.985981 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0327 23:10:00.985993 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=a48711a04cb7840d87fa43faff34344aebde39b8389a3b93f1f49af296359da5 I0327 23:10:00.986009 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=4f75aa3820e5cd64e75fc2b7fe727be2549d1819976800b6c0aa40fb7cd90c83 I0327 23:10:00.986023 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=ab9d84906af160ecea57ebefd08be054cf268b70e6fb7199171353db12963723 I0327 23:10:00.986046 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=0e191f19515f121a865e1125376772a19a166a24ad82bd8f28c101dde7c21c45 I0327 23:10:00.986055 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0327 23:10:00.986078 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=ba4380ab0957f32d75912a5518572daaea1974ebd3b062ddfadeeae939030062 I0327 23:10:00.986094 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0327 23:10:00.986103 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0327 23:10:00.986109 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.466090149s to process 39 records W0327 23:10:01.547437 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0327 23:10:02.547552 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0327 23:10:03.546415 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0327 23:10:03.546460 1 tasks_processing.go:74] worker 60 stopped. E0327 23:10:03.546476 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0327 23:10:03.546489 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0327 23:10:03.546500 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.027467111s to process 1 records I0327 23:10:11.445620 1 tasks_processing.go:74] worker 29 stopped. I0327 23:10:11.445665 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0327 23:10:11.445677 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.926572048s to process 1 records I0327 23:10:12.123927 1 tasks_processing.go:74] worker 14 stopped. I0327 23:10:12.124212 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=4aff5860ec073ddfdf1bec96303c96ef7753a0d476f8315de0a2062bde3a2fd5 I0327 23:10:12.124229 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.606046812s to process 1 records E0327 23:10:12.124290 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.606s with: function \"machines\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0327 23:10:12.125392 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0327 23:10:12.125407 1 periodic.go:209] Running workloads gatherer I0327 23:10:12.125422 1 tasks_processing.go:45] number of workers: 2 I0327 23:10:12.125428 1 tasks_processing.go:69] worker 1 listening for tasks. I0327 23:10:12.125432 1 tasks_processing.go:71] worker 1 working on workload_info task. I0327 23:10:12.125438 1 tasks_processing.go:69] worker 0 listening for tasks. I0327 23:10:12.125458 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0327 23:10:12.136202 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-65.ec2.internal... I0327 23:10:12.136211 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-196.ec2.internal... I0327 23:10:12.136224 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-41.ec2.internal... I0327 23:10:12.159522 1 tasks_processing.go:74] worker 0 stopped. I0327 23:10:12.159546 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 34.041304ms to process 0 records I0327 23:10:16.653674 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0327 23:10:16.677154 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0327 23:10:16.686104 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0327 23:10:16.696270 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0327 23:10:16.704852 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0327 23:10:16.714200 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (9ms) I0327 23:10:16.722697 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (8ms) I0327 23:10:16.732008 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (9ms) I0327 23:10:16.740101 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (8ms) I0327 23:10:16.748830 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (9ms) I0327 23:10:16.756931 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (8ms) I0327 23:10:16.766021 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (9ms) I0327 23:10:16.784572 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (19ms) I0327 23:10:16.886117 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (102ms) I0327 23:10:16.985960 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0327 23:10:17.085865 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (100ms) I0327 23:10:17.185820 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0327 23:10:17.285818 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (100ms) I0327 23:10:17.385623 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0327 23:10:17.486535 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (101ms) I0327 23:10:17.585086 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0327 23:10:17.687980 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (103ms) I0327 23:10:17.785705 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (98ms) I0327 23:10:17.887106 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0327 23:10:17.987187 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0327 23:10:18.086345 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0327 23:10:18.186813 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (100ms) I0327 23:10:18.285419 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (99ms) I0327 23:10:18.386422 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (101ms) I0327 23:10:18.486992 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (101ms) I0327 23:10:18.586015 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (99ms) I0327 23:10:18.685928 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0327 23:10:18.785063 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (99ms) I0327 23:10:18.886579 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (101ms) I0327 23:10:18.986209 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (100ms) I0327 23:10:19.086581 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0327 23:10:19.186325 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (100ms) I0327 23:10:19.288788 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (102ms) I0327 23:10:19.385757 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (97ms) I0327 23:10:19.486602 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (101ms) I0327 23:10:19.585335 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0327 23:10:19.685437 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0327 23:10:19.785882 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (100ms) I0327 23:10:19.886400 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (100ms) I0327 23:10:19.986525 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0327 23:10:20.087726 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (101ms) I0327 23:10:20.187096 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (99ms) I0327 23:10:20.285822 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (99ms) I0327 23:10:20.385549 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0327 23:10:20.487125 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (102ms) I0327 23:10:20.585910 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0327 23:10:20.585944 1 tasks_processing.go:74] worker 1 stopped. I0327 23:10:20.586236 1 recorder.go:75] Recording config/workload_info with fingerprint=d20f4e940cbdd48c7fb059f11b193b13b53aee75e68950774081a0319540ad60 I0327 23:10:20.586252 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.46050534s to process 1 records I0327 23:10:20.586268 1 periodic.go:256] Periodic gather workloads completed in 8.46s I0327 23:10:20.586278 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0327 23:10:20.586282 1 periodic.go:209] Running conditional gatherer I0327 23:10:20.591855 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0327 23:10:20.776372 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0327 23:10:20.778844 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.43.21:9091: connect: connection refused I0327 23:10:20.778926 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0327 23:10:20.784693 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0327 23:10:20.784709 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784715 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784719 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784722 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784725 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784728 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784731 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784734 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784737 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 23:10:20.784741 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0327 23:10:20.784754 1 tasks_processing.go:45] number of workers: 3 I0327 23:10:20.784760 1 tasks_processing.go:69] worker 2 listening for tasks. I0327 23:10:20.784764 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0327 23:10:20.784779 1 tasks_processing.go:69] worker 1 listening for tasks. I0327 23:10:20.784776 1 tasks_processing.go:69] worker 0 listening for tasks. I0327 23:10:20.784801 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0327 23:10:20.784803 1 tasks_processing.go:74] worker 0 stopped. I0327 23:10:20.784819 1 tasks_processing.go:71] worker 1 working on remote_configuration task. I0327 23:10:20.784841 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0327 23:10:20.784851 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 855ns to process 1 records I0327 23:10:20.784884 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0327 23:10:20.784908 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.394µs to process 1 records I0327 23:10:20.784913 1 tasks_processing.go:74] worker 1 stopped. I0327 23:10:20.785086 1 tasks_processing.go:74] worker 2 stopped. I0327 23:10:20.785102 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 276.594µs to process 0 records I0327 23:10:20.785125 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0327 23:10:20.785151 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0327 23:10:20.814952 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=311826d5a4d449258226c4056dc8850b77a4bb9749b1aca76d2f694fb0b5c067 I0327 23:10:20.815135 1 diskrecorder.go:70] Writing 111 records to /var/lib/insights-operator/insights-2026-03-27-231020.tar.gz I0327 23:10:20.821871 1 diskrecorder.go:51] Wrote 111 records to disk in 6ms I0327 23:10:20.821921 1 periodic.go:278] Gathering cluster info every 2h0m0s I0327 23:10:20.821942 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0327 23:11:28.518292 1 diskrecorder.go:223] Found files to send: insights-2026-03-27-231020.tar.gz I0327 23:11:28.518324 1 insightsuploader.go:150] Checking archives to upload periodically every 15m20.42832179s I0327 23:11:28.518334 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0327 23:11:28.528606 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0327 23:11:28.650570 1 requests.go:87] Successfully reported id=2026-03-27T23:11:28Z x-rh-insights-request-id=34f3a6112bf749a3a1000a1f5504c737, wrote=48269 I0327 23:11:28.650635 1 insightsuploader.go:187] Uploaded report successfully in 132.292209ms I0327 23:11:28.650659 1 controller.go:128] Initializing last reported time to 2026-03-27T23:11:28Z I0327 23:11:28.650676 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0327 23:11:28.650682 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0327 23:11:28.650690 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0327 23:11:28.657064 1 controller.go:512] The operator is healthy I0327 23:11:58.551964 1 controller.go:512] The operator is healthy I0327 23:12:34.631217 1 insightsreport.go:137] Pulling report from smart-proxy I0327 23:12:34.631241 1 insightsreport.go:149] Retrieving report I0327 23:12:34.638257 1 requests.go:110] Retrieving report for cluster: 415e7610-beff-4d6a-b63f-b6beb37fa22e I0327 23:12:34.638269 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/415e7610-beff-4d6a-b63f-b6beb37fa22e/reports I0327 23:12:34.642742 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/415e7610-beff-4d6a-b63f-b6beb37fa22e/reports I0327 23:12:34.807529 1 insightsreport.go:184] Report retrieved I0327 23:12:34.832381 1 insightsreport.go:239] Report retrieved correctly I0327 23:13:58.553430 1 controller.go:512] The operator is healthy I0327 23:14:58.510830 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:14:58.517497 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:14:58.517519 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:14:58.521677 1 secretconfigobserver.go:119] support secret does not exist I0327 23:15:58.552691 1 controller.go:512] The operator is healthy I0327 23:17:58.553235 1 controller.go:512] The operator is healthy I0327 23:19:58.508402 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:19:58.522451 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:19:58.523702 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:19:58.530057 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:19:58.530077 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:19:58.534219 1 secretconfigobserver.go:119] support secret does not exist I0327 23:19:58.556489 1 controller.go:512] The operator is healthy I0327 23:21:58.553413 1 controller.go:512] The operator is healthy I0327 23:23:58.554357 1 controller.go:512] The operator is healthy I0327 23:24:58.534314 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:24:58.542590 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:24:58.542611 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:24:58.546970 1 secretconfigobserver.go:119] support secret does not exist I0327 23:25:58.553439 1 controller.go:512] The operator is healthy I0327 23:26:54.079830 1 insightsuploader.go:146] Nothing to report since 2026-03-27T23:11:28Z I0327 23:27:58.553056 1 controller.go:512] The operator is healthy I0327 23:29:58.508987 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:29:58.513338 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:29:58.547499 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:29:58.552032 1 controller.go:512] The operator is healthy I0327 23:29:58.552351 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:29:58.552366 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:29:58.556429 1 secretconfigobserver.go:119] support secret does not exist I0327 23:31:58.552705 1 controller.go:512] The operator is healthy I0327 23:33:58.553096 1 controller.go:512] The operator is healthy I0327 23:34:58.557449 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:34:58.563469 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:34:58.563491 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:34:58.569451 1 secretconfigobserver.go:119] support secret does not exist I0327 23:35:58.553362 1 controller.go:512] The operator is healthy I0327 23:37:58.552974 1 controller.go:512] The operator is healthy I0327 23:39:58.510047 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:39:58.514452 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:39:58.551840 1 controller.go:512] The operator is healthy I0327 23:39:58.570005 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:39:58.574572 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:39:58.574589 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:39:58.578624 1 secretconfigobserver.go:119] support secret does not exist I0327 23:41:58.552631 1 controller.go:512] The operator is healthy I0327 23:43:06.054629 1 insightsuploader.go:146] Nothing to report since 2026-03-27T23:11:28Z I0327 23:43:58.554070 1 controller.go:512] The operator is healthy I0327 23:44:58.578720 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:44:58.584370 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:44:58.584390 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:44:58.588582 1 secretconfigobserver.go:119] support secret does not exist I0327 23:45:58.553270 1 controller.go:512] The operator is healthy I0327 23:47:58.552506 1 controller.go:512] The operator is healthy I0327 23:49:58.509976 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 23:49:58.552402 1 controller.go:512] The operator is healthy I0327 23:49:58.588649 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0327 23:49:58.593035 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 23:49:58.593057 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0327 23:49:58.596919 1 secretconfigobserver.go:119] support secret does not exist I0327 23:51:58.552705 1 controller.go:512] The operator is healthy