I0415 22:54:15.795305 1 cmd.go:253] Using service-serving-cert provided certificates I0415 22:54:15.795712 1 observer_polling.go:159] Starting file observer I0415 22:54:16.311464 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0415 22:54:16.311654 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0415 22:54:16.312234 1 secure_serving.go:57] Forcing use of http/1.1 only W0415 22:54:16.312255 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0415 22:54:16.312259 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0415 22:54:16.312263 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0415 22:54:16.312265 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0415 22:54:16.312268 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0415 22:54:16.312270 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0415 22:54:16.312279 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0415 22:54:16.315548 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0415 22:54:16.315577 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"c0775c6d-c9cb-4cf9-9d37-f010b4181d7f", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0415 22:54:16.317629 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0415 22:54:16.317644 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0415 22:54:16.317645 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0415 22:54:16.317659 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0415 22:54:16.317973 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0415 22:54:16.318090 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0415 22:54:16.318876 1 secure_serving.go:213] Serving securely on [::]:8443 I0415 22:54:16.318926 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" I0415 22:54:16.319147 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" W0415 22:54:16.327270 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0415 22:54:16.327294 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0415 22:54:16.327389 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0415 22:54:16.331847 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 22:54:16.331867 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0415 22:54:16.336179 1 secretconfigobserver.go:119] support secret does not exist I0415 22:54:16.340555 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 22:54:16.344580 1 secretconfigobserver.go:119] support secret does not exist I0415 22:54:16.348215 1 recorder.go:161] Pruning old reports every 4h12m33s, max age is 288h0m0s I0415 22:54:16.352877 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0415 22:54:16.352885 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0415 22:54:16.352895 1 insightsreport.go:296] Starting report retriever I0415 22:54:16.352894 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0415 22:54:16.352902 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0415 22:54:16.352913 1 periodic.go:209] Running clusterconfig gatherer I0415 22:54:16.352958 1 tasks_processing.go:45] number of workers: 64 I0415 22:54:16.352988 1 tasks_processing.go:69] worker 7 listening for tasks. I0415 22:54:16.352996 1 tasks_processing.go:69] worker 15 listening for tasks. I0415 22:54:16.353017 1 tasks_processing.go:69] worker 8 listening for tasks. I0415 22:54:16.353019 1 tasks_processing.go:69] worker 4 listening for tasks. I0415 22:54:16.353023 1 tasks_processing.go:69] worker 9 listening for tasks. I0415 22:54:16.353024 1 tasks_processing.go:69] worker 3 listening for tasks. I0415 22:54:16.353028 1 tasks_processing.go:69] worker 10 listening for tasks. I0415 22:54:16.353029 1 tasks_processing.go:69] worker 6 listening for tasks. I0415 22:54:16.353032 1 tasks_processing.go:69] worker 0 listening for tasks. I0415 22:54:16.353034 1 tasks_processing.go:69] worker 11 listening for tasks. I0415 22:54:16.353034 1 tasks_processing.go:71] worker 10 working on ceph_cluster task. I0415 22:54:16.353038 1 tasks_processing.go:69] worker 1 listening for tasks. I0415 22:54:16.353039 1 tasks_processing.go:69] worker 12 listening for tasks. I0415 22:54:16.353046 1 tasks_processing.go:69] worker 2 listening for tasks. I0415 22:54:16.353036 1 tasks_processing.go:71] worker 6 working on metrics task. I0415 22:54:16.353050 1 tasks_processing.go:69] worker 14 listening for tasks. I0415 22:54:16.353052 1 tasks_processing.go:69] worker 42 listening for tasks. I0415 22:54:16.353063 1 tasks_processing.go:69] worker 16 listening for tasks. I0415 22:54:16.353065 1 tasks_processing.go:69] worker 43 listening for tasks. I0415 22:54:16.353068 1 tasks_processing.go:69] worker 58 listening for tasks. I0415 22:54:16.353073 1 tasks_processing.go:69] worker 17 listening for tasks. I0415 22:54:16.353073 1 tasks_processing.go:69] worker 36 listening for tasks. I0415 22:54:16.353077 1 tasks_processing.go:69] worker 33 listening for tasks. I0415 22:54:16.353072 1 tasks_processing.go:69] worker 30 listening for tasks. I0415 22:54:16.353080 1 tasks_processing.go:69] worker 34 listening for tasks. I0415 22:54:16.353083 1 tasks_processing.go:69] worker 37 listening for tasks. I0415 22:54:16.353085 1 tasks_processing.go:69] worker 32 listening for tasks. I0415 22:54:16.353055 1 tasks_processing.go:69] worker 28 listening for tasks. I0415 22:54:16.353057 1 tasks_processing.go:69] worker 35 listening for tasks. I0415 22:54:16.353091 1 tasks_processing.go:71] worker 8 working on aggregated_monitoring_cr_names task. I0415 22:54:16.353057 1 tasks_processing.go:69] worker 59 listening for tasks. I0415 22:54:16.353057 1 tasks_processing.go:69] worker 56 listening for tasks. I0415 22:54:16.353064 1 tasks_processing.go:69] worker 57 listening for tasks. I0415 22:54:16.353063 1 tasks_processing.go:69] worker 29 listening for tasks. I0415 22:54:16.353099 1 tasks_processing.go:69] worker 39 listening for tasks. I0415 22:54:16.353077 1 tasks_processing.go:69] worker 44 listening for tasks. I0415 22:54:16.353081 1 tasks_processing.go:69] worker 45 listening for tasks. I0415 22:54:16.353103 1 tasks_processing.go:71] worker 9 working on silenced_alerts task. I0415 22:54:16.353111 1 tasks_processing.go:69] worker 46 listening for tasks. I0415 22:54:16.353108 1 tasks_processing.go:69] worker 31 listening for tasks. I0415 22:54:16.353114 1 tasks_processing.go:69] worker 41 listening for tasks. I0415 22:54:16.353111 1 tasks_processing.go:69] worker 60 listening for tasks. I0415 22:54:16.353119 1 tasks_processing.go:71] worker 45 working on openstack_dataplanenodesets task. I0415 22:54:16.353125 1 tasks_processing.go:69] worker 52 listening for tasks. I0415 22:54:16.353127 1 tasks_processing.go:69] worker 62 listening for tasks. I0415 22:54:16.353130 1 tasks_processing.go:69] worker 54 listening for tasks. I0415 22:54:16.353132 1 tasks_processing.go:69] worker 20 listening for tasks. I0415 22:54:16.353136 1 tasks_processing.go:71] worker 62 working on service_accounts task. I0415 22:54:16.353140 1 tasks_processing.go:69] worker 55 listening for tasks. I0415 22:54:16.353142 1 tasks_processing.go:71] worker 7 working on ingress task. I0415 22:54:16.353143 1 tasks_processing.go:71] worker 31 working on dvo_metrics task. I0415 22:54:16.353145 1 tasks_processing.go:69] worker 27 listening for tasks. I0415 22:54:16.353082 1 tasks_processing.go:69] worker 18 listening for tasks. I0415 22:54:16.353172 1 tasks_processing.go:71] worker 58 working on version task. I0415 22:54:16.353176 1 tasks_processing.go:71] worker 16 working on nodes task. I0415 22:54:16.353179 1 tasks_processing.go:71] worker 39 working on image_registries task. I0415 22:54:16.353181 1 tasks_processing.go:71] worker 57 working on machine_healthchecks task. I0415 22:54:16.353183 1 tasks_processing.go:71] worker 55 working on ingress_certificates task. I0415 22:54:16.353186 1 tasks_processing.go:71] worker 54 working on proxies task. I0415 22:54:16.353184 1 tasks_processing.go:71] worker 33 working on config_maps task. I0415 22:54:16.353046 1 tasks_processing.go:69] worker 13 listening for tasks. I0415 22:54:16.353221 1 tasks_processing.go:71] worker 13 working on sap_datahubs task. I0415 22:54:16.353180 1 tasks_processing.go:71] worker 56 working on sap_pods task. I0415 22:54:16.353123 1 tasks_processing.go:71] worker 41 working on support_secret task. I0415 22:54:16.353135 1 tasks_processing.go:69] worker 51 listening for tasks. I0415 22:54:16.353108 1 tasks_processing.go:69] worker 40 listening for tasks. I0415 22:54:16.353112 1 tasks_processing.go:71] worker 4 working on sap_config task. I0415 22:54:16.353470 1 tasks_processing.go:71] worker 51 working on active_alerts task. I0415 22:54:16.353140 1 tasks_processing.go:69] worker 21 listening for tasks. I0415 22:54:16.353119 1 tasks_processing.go:69] worker 61 listening for tasks. I0415 22:54:16.353121 1 tasks_processing.go:69] worker 53 listening for tasks. I0415 22:54:16.353120 1 tasks_processing.go:69] worker 19 listening for tasks. I0415 22:54:16.353125 1 tasks_processing.go:69] worker 48 listening for tasks. I0415 22:54:16.353128 1 tasks_processing.go:71] worker 60 working on qemu_kubevirt_launcher_logs task. I0415 22:54:16.353530 1 tasks_processing.go:71] worker 48 working on cluster_apiserver task. I0415 22:54:16.353567 1 tasks_processing.go:71] worker 61 working on oauths task. I0415 22:54:16.353594 1 tasks_processing.go:71] worker 40 working on networks task. I0415 22:54:16.353594 1 tasks_processing.go:71] worker 53 working on machine_config_pools task. I0415 22:54:16.353596 1 tasks_processing.go:71] worker 19 working on install_plans task. I0415 22:54:16.353117 1 tasks_processing.go:71] worker 46 working on cost_management_metrics_configs task. I0415 22:54:16.353134 1 tasks_processing.go:71] worker 52 working on openshift_logging task. I0415 22:54:16.353135 1 tasks_processing.go:69] worker 24 listening for tasks. I0415 22:54:16.353132 1 tasks_processing.go:69] worker 47 listening for tasks. I0415 22:54:16.353143 1 tasks_processing.go:69] worker 26 listening for tasks. I0415 22:54:16.353146 1 tasks_processing.go:69] worker 22 listening for tasks. I0415 22:54:16.353149 1 tasks_processing.go:71] worker 3 working on clusterroles task. I0415 22:54:16.353876 1 tasks_processing.go:71] worker 24 working on operators task. I0415 22:54:16.353994 1 tasks_processing.go:71] worker 22 working on machine_configs task. I0415 22:54:16.354047 1 tasks_processing.go:71] worker 47 working on authentication task. I0415 22:54:16.354066 1 tasks_processing.go:71] worker 26 working on monitoring_persistent_volumes task. I0415 22:54:16.353150 1 tasks_processing.go:69] worker 23 listening for tasks. I0415 22:54:16.354507 1 tasks_processing.go:71] worker 23 working on overlapping_namespace_uids task. I0415 22:54:16.353093 1 tasks_processing.go:69] worker 38 listening for tasks. I0415 22:54:16.353152 1 tasks_processing.go:69] worker 25 listening for tasks. I0415 22:54:16.353158 1 tasks_processing.go:71] worker 14 working on machines task. I0415 22:54:16.353150 1 tasks_processing.go:69] worker 50 listening for tasks. I0415 22:54:16.353158 1 tasks_processing.go:71] worker 37 working on storage_classes task. I0415 22:54:16.353162 1 tasks_processing.go:71] worker 0 working on certificate_signing_requests task. I0415 22:54:16.354852 1 tasks_processing.go:71] worker 50 working on node_logs task. I0415 22:54:16.353161 1 tasks_processing.go:71] worker 28 working on pod_network_connectivity_checks task. I0415 22:54:16.354979 1 tasks_processing.go:71] worker 38 working on storage_cluster task. I0415 22:54:16.353161 1 tasks_processing.go:71] worker 32 working on lokistack task. I0415 22:54:16.353162 1 tasks_processing.go:71] worker 30 working on mutating_webhook_configurations task. I0415 22:54:16.353163 1 tasks_processing.go:71] worker 34 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0415 22:54:16.353164 1 tasks_processing.go:71] worker 15 working on container_runtime_configs task. I0415 22:54:16.353167 1 tasks_processing.go:71] worker 11 working on openstack_controlplanes task. I0415 22:54:16.353167 1 tasks_processing.go:71] worker 43 working on machine_autoscalers task. I0415 22:54:16.353169 1 tasks_processing.go:71] worker 12 working on nodenetworkstates task. I0415 22:54:16.353172 1 tasks_processing.go:71] worker 42 working on jaegers task. I0415 22:54:16.353171 1 tasks_processing.go:71] worker 36 working on tsdb_status task. I0415 22:54:16.353171 1 tasks_processing.go:71] worker 2 working on infrastructures task. I0415 22:54:16.353171 1 tasks_processing.go:71] worker 17 working on crds task. I0415 22:54:16.353173 1 tasks_processing.go:71] worker 1 working on image_pruners task. I0415 22:54:16.353079 1 tasks_processing.go:69] worker 63 listening for tasks. I0415 22:54:16.353177 1 tasks_processing.go:71] worker 59 working on machine_sets task. I0415 22:54:16.353183 1 tasks_processing.go:71] worker 20 working on image task. I0415 22:54:16.353183 1 tasks_processing.go:71] worker 18 working on openshift_machine_api_events task. I0415 22:54:16.353180 1 tasks_processing.go:71] worker 44 working on nodenetworkconfigurationpolicies task. I0415 22:54:16.353183 1 tasks_processing.go:71] worker 29 working on openstack_version task. I0415 22:54:16.353029 1 tasks_processing.go:69] worker 5 listening for tasks. I0415 22:54:16.353187 1 tasks_processing.go:71] worker 27 working on olm_operators task. I0415 22:54:16.353695 1 tasks_processing.go:71] worker 21 working on schedulers task. I0415 22:54:16.353156 1 tasks_processing.go:71] worker 35 working on feature_gates task. I0415 22:54:16.353153 1 tasks_processing.go:69] worker 49 listening for tasks. I0415 22:54:16.355031 1 tasks_processing.go:71] worker 25 working on validating_webhook_configurations task. I0415 22:54:16.355462 1 tasks_processing.go:71] worker 63 working on operators_pods_and_events task. I0415 22:54:16.355856 1 tasks_processing.go:71] worker 10 working on container_images task. I0415 22:54:16.356852 1 tasks_processing.go:71] worker 49 working on openstack_dataplanedeployments task. I0415 22:54:16.357354 1 tasks_processing.go:71] worker 5 working on pdbs task. I0415 22:54:16.356818 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 2.801034ms to process 0 records I0415 22:54:16.358727 1 controller.go:128] Initializing last reported time to 2026-04-15T22:54:11Z I0415 22:54:16.358744 1 controller.go:326] The initial operator extension status is healthy I0415 22:54:16.358755 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0415 22:54:16.358760 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0415 22:54:16.358763 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0415 22:54:16.358777 1 controller.go:489] The operator is still being initialized I0415 22:54:16.358782 1 controller.go:512] The operator is healthy I0415 22:54:16.359159 1 tasks_processing.go:74] worker 57 stopped. E0415 22:54:16.359170 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0415 22:54:16.359179 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 5.961875ms to process 0 records E0415 22:54:16.360362 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.34.124:9091: connect: connection refused I0415 22:54:16.360375 1 tasks_processing.go:74] worker 59 stopped. E0415 22:54:16.360374 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.34.124:9091: connect: connection refused I0415 22:54:16.360387 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 5.184626ms to process 0 records E0415 22:54:16.360397 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.34.124:9091: connect: connection refused I0415 22:54:16.360411 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 5.025611ms to process 0 records E0415 22:54:16.360422 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.34.124:9091: connect: connection refused I0415 22:54:16.360445 1 tasks_processing.go:74] worker 36 stopped. I0415 22:54:16.360447 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 7.33282ms to process 0 records I0415 22:54:16.360452 1 tasks_processing.go:74] worker 6 stopped. I0415 22:54:16.361449 1 tasks_processing.go:74] worker 28 stopped. E0415 22:54:16.361461 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0415 22:54:16.361468 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 6.511308ms to process 0 records I0415 22:54:16.361977 1 tasks_processing.go:74] worker 42 stopped. I0415 22:54:16.361992 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 6.619778ms to process 0 records I0415 22:54:16.362758 1 tasks_processing.go:74] worker 4 stopped. I0415 22:54:16.362770 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 9.318381ms to process 0 records I0415 22:54:16.362818 1 tasks_processing.go:74] worker 44 stopped. I0415 22:54:16.362830 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 7.639345ms to process 0 records I0415 22:54:16.364490 1 tasks_processing.go:74] worker 52 stopped. I0415 22:54:16.364505 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 10.735764ms to process 0 records I0415 22:54:16.364646 1 tasks_processing.go:74] worker 45 stopped. I0415 22:54:16.364655 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 11.511972ms to process 0 records I0415 22:54:16.365471 1 tasks_processing.go:74] worker 11 stopped. I0415 22:54:16.365480 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 10.364648ms to process 0 records I0415 22:54:16.365897 1 tasks_processing.go:74] worker 14 stopped. E0415 22:54:16.365910 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0415 22:54:16.365921 1 gather.go:177] gatherer "clusterconfig" function "machines" took 11.347114ms to process 0 records I0415 22:54:16.366136 1 tasks_processing.go:74] worker 54 stopped. I0415 22:54:16.366211 1 recorder.go:75] Recording config/proxy with fingerprint=61914b53a83563647d11f03cf3bcb70a15dac8d5e505ad1a595151e79584e427 I0415 22:54:16.366226 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 12.935899ms to process 1 records I0415 22:54:16.366291 1 tasks_processing.go:74] worker 38 stopped. I0415 22:54:16.366302 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 11.039971ms to process 0 records I0415 22:54:16.366688 1 tasks_processing.go:74] worker 37 stopped. I0415 22:54:16.366767 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=65fcc24c34338f9acc3901acf9e23b7f0684b51b79441f40d0ce56297b49e992 I0415 22:54:16.366786 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=76319b45883f53f84b7e8a1367c46b3c8704c9262224d8e171cadcd9b864782b I0415 22:54:16.366793 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 12.071219ms to process 2 records I0415 22:54:16.369576 1 tasks_processing.go:74] worker 46 stopped. I0415 22:54:16.369584 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 15.86102ms to process 0 records I0415 22:54:16.369593 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 16.349594ms to process 0 records I0415 22:54:16.369598 1 tasks_processing.go:74] worker 56 stopped. I0415 22:54:16.370119 1 tasks_processing.go:74] worker 43 stopped. I0415 22:54:16.370130 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 14.346035ms to process 0 records I0415 22:54:16.370366 1 tasks_processing.go:74] worker 12 stopped. I0415 22:54:16.370383 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 14.571872ms to process 0 records I0415 22:54:16.371493 1 tasks_processing.go:74] worker 7 stopped. I0415 22:54:16.371682 1 recorder.go:75] Recording config/ingress with fingerprint=7e8604a2e83ecc221eaf8298f989ecede251129e1457d53e55ae530114e06b52 I0415 22:54:16.371698 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 18.341618ms to process 1 records I0415 22:54:16.371742 1 tasks_processing.go:74] worker 35 stopped. I0415 22:54:16.371828 1 recorder.go:75] Recording config/featuregate with fingerprint=935ce2542b53611397c7045c6acd0f9a3f16bf28677cfb385748b212bef085b3 I0415 22:54:16.371843 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 15.749989ms to process 1 records I0415 22:54:16.371923 1 tasks_processing.go:74] worker 40 stopped. I0415 22:54:16.371948 1 recorder.go:75] Recording config/network with fingerprint=6c2aa3660124b7e91cb4b247e020d724163c4cf3d6d6250653f6c6beed3edc46 I0415 22:54:16.371957 1 gather.go:177] gatherer "clusterconfig" function "networks" took 18.146926ms to process 1 records I0415 22:54:16.372020 1 tasks_processing.go:74] worker 61 stopped. I0415 22:54:16.372125 1 recorder.go:75] Recording config/oauth with fingerprint=d28da4b721444f34caef1f02303e07f6fea6f5ec296fc2b6c89fc30b0cb7a56b I0415 22:54:16.372133 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 18.202443ms to process 1 records I0415 22:54:16.372141 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 16.380629ms to process 0 records I0415 22:54:16.372147 1 tasks_processing.go:74] worker 15 stopped. I0415 22:54:16.374479 1 tasks_processing.go:74] worker 47 stopped. I0415 22:54:16.374645 1 recorder.go:75] Recording config/authentication with fingerprint=d1896e48b3404cd4bcbea013bec5786d1111adebd28e3632fb3a7c5ad9b2158c I0415 22:54:16.374656 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 20.378646ms to process 1 records I0415 22:54:16.374737 1 tasks_processing.go:74] worker 1 stopped. I0415 22:54:16.374854 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=c232c73839f5b79ab96c957c14b7140428af94d691fe73a8735b49dd1caf14db I0415 22:54:16.374866 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 19.160886ms to process 1 records I0415 22:54:16.378940 1 tasks_processing.go:74] worker 13 stopped. I0415 22:54:16.378953 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 25.708155ms to process 0 records I0415 22:54:16.379028 1 tasks_processing.go:74] worker 26 stopped. I0415 22:54:16.379036 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 24.559337ms to process 0 records I0415 22:54:16.379065 1 tasks_processing.go:74] worker 29 stopped. I0415 22:54:16.379085 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 23.079138ms to process 0 records I0415 22:54:16.383242 1 gather_logs.go:145] no pods in namespace were found I0415 22:54:16.383260 1 tasks_processing.go:74] worker 60 stopped. I0415 22:54:16.383272 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 29.73445ms to process 0 records I0415 22:54:16.383392 1 tasks_processing.go:74] worker 32 stopped. I0415 22:54:16.383404 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 28.001781ms to process 0 records I0415 22:54:16.383469 1 tasks_processing.go:74] worker 49 stopped. I0415 22:54:16.383484 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 26.554485ms to process 0 records I0415 22:54:16.383580 1 tasks_processing.go:74] worker 41 stopped. E0415 22:54:16.383589 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0415 22:54:16.383599 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 30.339569ms to process 0 records I0415 22:54:16.384079 1 tasks_processing.go:74] worker 18 stopped. I0415 22:54:16.384100 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 28.081146ms to process 0 records I0415 22:54:16.384291 1 tasks_processing.go:74] worker 48 stopped. I0415 22:54:16.384459 1 recorder.go:75] Recording config/apiserver with fingerprint=c97ceee03ce209976cc34f135581323c21c1384648c4595aa23df71492e0ad87 I0415 22:54:16.384481 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 30.748023ms to process 1 records I0415 22:54:16.384920 1 tasks_processing.go:74] worker 21 stopped. I0415 22:54:16.385552 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=834d7a9af8e127e93efdbda546b6fd38c435e9fbe9f2d0d3687c3609acc86761 I0415 22:54:16.385592 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 28.927299ms to process 1 records I0415 22:54:16.385811 1 tasks_processing.go:74] worker 58 stopped. I0415 22:54:16.386148 1 recorder.go:75] Recording config/version with fingerprint=444e40b7bfd62bc2ff5c9f4c49f013e9dfe8f833a1343e655ba44befee4d008f I0415 22:54:16.386215 1 recorder.go:75] Recording config/id with fingerprint=e2ebf3d935ca72c7b51780936fc8568338ccd0c3fd301d897e94692d43aeb94f I0415 22:54:16.386231 1 gather.go:177] gatherer "clusterconfig" function "version" took 32.269992ms to process 2 records I0415 22:54:16.386390 1 recorder.go:75] Recording config/image with fingerprint=2f696acb21b053438290b072cde928686fb97d61ae7c1616d724a10280ab180f I0415 22:54:16.386405 1 gather.go:177] gatherer "clusterconfig" function "image" took 30.605897ms to process 1 records I0415 22:54:16.386427 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0415 22:54:16.386451 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 31.42887ms to process 1 records I0415 22:54:16.386499 1 tasks_processing.go:74] worker 23 stopped. I0415 22:54:16.386512 1 tasks_processing.go:74] worker 20 stopped. I0415 22:54:16.388613 1 tasks_processing.go:74] worker 2 stopped. W0415 22:54:16.388786 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0415 22:54:16.389203 1 recorder.go:75] Recording config/infrastructure with fingerprint=88bc932d359d530a46dd30000e10f6210c82bc2ba076da21ec94e2e8f1a8d5d3 I0415 22:54:16.389226 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 32.841082ms to process 1 records I0415 22:54:16.389505 1 tasks_processing.go:74] worker 16 stopped. I0415 22:54:16.389892 1 recorder.go:75] Recording config/node/ip-10-0-0-226.ec2.internal with fingerprint=041f5441a08c5f5762ebe324d93a28193d19dfd2a1a2522c4ed750c962bfd8d0 I0415 22:54:16.389982 1 recorder.go:75] Recording config/node/ip-10-0-1-214.ec2.internal with fingerprint=4e17b91d80041ff56f81c35da37341e2fc47c1439ca2a8d5077c08aad438db46 I0415 22:54:16.390086 1 recorder.go:75] Recording config/node/ip-10-0-2-197.ec2.internal with fingerprint=81ba55822cc52ef63392ac5511fa03597c9fe29401c218f1a4a3f14c4607638f I0415 22:54:16.390099 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 36.31736ms to process 3 records I0415 22:54:16.390106 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 36.119266ms to process 0 records I0415 22:54:16.390132 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 34.956213ms to process 0 records I0415 22:54:16.390154 1 tasks_processing.go:74] worker 53 stopped. I0415 22:54:16.390166 1 tasks_processing.go:74] worker 50 stopped. I0415 22:54:16.390237 1 tasks_processing.go:74] worker 30 stopped. I0415 22:54:16.390342 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=47b0e33086285ea06e8e6b2c075f8be73ee26cd4c7efea04f4207da9036a26ee I0415 22:54:16.390389 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=c366905d7eebc5ee43dbed5bcbe08c205857ba2355ee11b1e11cfd3e163fd713 I0415 22:54:16.390430 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=8b977109e343e5c263dd1d816192a66c5787fed4c1c7564f26b9cca228662804 I0415 22:54:16.390445 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 34.252613ms to process 3 records I0415 22:54:16.390457 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 35.181489ms to process 0 records I0415 22:54:16.390465 1 tasks_processing.go:74] worker 0 stopped. I0415 22:54:16.390534 1 tasks_processing.go:74] worker 39 stopped. I0415 22:54:16.390930 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=2def040590b4c6ca14a25ffb07a2ff0f77b64060d9b02ae89ac87abfa040edcc I0415 22:54:16.390944 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 37.345888ms to process 1 records I0415 22:54:16.391028 1 tasks_processing.go:74] worker 25 stopped. I0415 22:54:16.391048 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=96f41fb7167edc8643b4abc0eb2f455be8c6c439083dec05ba31ccd268e9b646 I0415 22:54:16.391105 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=0e8d697232272291e6bc5ccc3d7fcb533aabb955c2ae2589d9158f461a1dcfd3 I0415 22:54:16.391218 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=3e673686641de016c813946128ae4a4ed623f7bc5558ca62eda9cf2e0953a2b3 I0415 22:54:16.391255 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=038a2d043a956c38d597382d81261a2ccba5e28c0b5935eb242d9c82f803c984 I0415 22:54:16.391283 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=e8b618dcd992aff983d1e1026560d30e31f5afc0f2fd170cc897f237065c3ffc I0415 22:54:16.391321 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=7efb2dc10f284e250866f51568ca08d426e251fe1fed19dd3cd93f02a47f89cb I0415 22:54:16.391351 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=884efae586da9c80258708a78e68b8d6e1056e517efef25b6a8af1fb3f7b461e I0415 22:54:16.391377 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=89b8beeca8be4aac1e53a112ef4244527ee635e9240c237492349ff217b523c6 I0415 22:54:16.391401 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=681bf1ad7f8531724e73e3acf0e2aea87cff7c09472e29c8f922e34e12acf6e1 I0415 22:54:16.391435 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=8ce967bd5f9c5834ebc1093c6b19e08061602b8b387e4fbc3b48f974cd6ae4c8 I0415 22:54:16.391460 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=0697112aa640ebd42a9ca1d6ed265f8b6975231cc1d0514d17ef1c17d8f739fb I0415 22:54:16.391481 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=dd4e76c9028e1361ced600d44542adaa0c6d4ad909a4b18942839f1a5d4d67a7 I0415 22:54:16.391507 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=4b97bb8dc998f5e903a89322965521731ea21a0e02253414af4787d15f9f20fb I0415 22:54:16.391516 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 35.585703ms to process 13 records I0415 22:54:16.391618 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=874d731d8a338bb20b98be012ddda20f6e33ee82683acae4de8e818fa1c4a543 I0415 22:54:16.391639 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=802e9c57eecbe8072afcfbb9d8661a326c2d8da4009a4f326d8d8ebad8ec1775 I0415 22:54:16.391652 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=ed2047abb320b9128cd07365e59b70442b1cc21d1dc574c52a16325c1e0f28d3 I0415 22:54:16.391664 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=70e14098835d3f16af5d87266e4f4c4fd8acc0945d4c8befb99d333a3ccd54e5 I0415 22:54:16.391682 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=ec9425a82c5ae8937a7884449ec9f88ba6a2311dce891b295006659c844e03f6 I0415 22:54:16.391617 1 tasks_processing.go:74] worker 5 stopped. I0415 22:54:16.391925 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=ae678e8694db5a8b23625c87b49a6220f59642ed8876ea320d04e8fa882bf716 I0415 22:54:16.392018 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=6acc7fe142ebb795ce3021ad3da32dbdf0af530ec664630618388194cd5f3088 I0415 22:54:16.392140 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=905c784a26d830f5695c887996ad393c9544d465c89688734aa5a2c5d5edf13a I0415 22:54:16.392168 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=bb3dec6671aeac11f03d327b6b65f867e5262c5a807c8670bd67ce4b6a21e71a I0415 22:54:16.392203 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=203bc1a3077ecd42d9cb29b94161b13ede42b3f2bbad8857c50d309285374101 I0415 22:54:16.392267 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=c3a2c9110c4dddb814eba56a00e1663cf501ebe83ed6d1d32ab5746ec513ef55 I0415 22:54:16.392312 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 33.767198ms to process 11 records I0415 22:54:16.392655 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=f8f21672f1fd6332279dca2843f04f8e99a296589729fc62f78b6afa7053bc8b I0415 22:54:16.392880 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=8179c1e67d96c9b2e9ac34aa483c6318313f6c440488c019f95bb842653badd8 I0415 22:54:16.392925 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 37.860846ms to process 2 records I0415 22:54:16.392940 1 tasks_processing.go:74] worker 3 stopped. I0415 22:54:16.393958 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0415 22:54:16.393963 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0415 22:54:16.394257 1 operator.go:288] started I0415 22:54:16.394291 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0415 22:54:16.395740 1 tasks_processing.go:74] worker 63 stopped. I0415 22:54:16.395756 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 40.261167ms to process 0 records I0415 22:54:16.401525 1 tasks_processing.go:74] worker 17 stopped. I0415 22:54:16.402122 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=f5df16972f085adb3215aff538891c252f22d6328e4309ba00ee9a1cc56d88f6 I0415 22:54:16.402400 1 controller.go:212] Source scaController *sca.Controller is not ready I0415 22:54:16.402410 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0415 22:54:16.402414 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0415 22:54:16.402417 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0415 22:54:16.402419 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0415 22:54:16.402454 1 controller.go:489] The operator is still being initialized I0415 22:54:16.402462 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=29b0ac790a830c24370ce1dd8a5e5dd3f723f61a14b3c30030f2b2b371e3d346 I0415 22:54:16.402477 1 gather.go:177] gatherer "clusterconfig" function "crds" took 46.40186ms to process 2 records I0415 22:54:16.402465 1 controller.go:512] The operator is healthy I0415 22:54:16.402735 1 tasks_processing.go:74] worker 27 stopped. I0415 22:54:16.402826 1 recorder.go:75] Recording config/olm_operators with fingerprint=4b155d0cb62b8de88af6efdd61e91446e3fe7cd8a67c7447538bb1b194d18d85 I0415 22:54:16.402840 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 46.744153ms to process 1 records I0415 22:54:16.408383 1 tasks_processing.go:74] worker 8 stopped. I0415 22:54:16.408405 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 55.279458ms to process 0 records I0415 22:54:16.417242 1 tasks_processing.go:74] worker 10 stopped. I0415 22:54:16.417676 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0415 22:54:16.417681 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0415 22:54:16.418365 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0415 22:54:16.418568 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-fj7cf with fingerprint=eca0b78a77cb247554a86ce1613bf1cad5883423f8dfa302a15a85a9b42c1ae8 I0415 22:54:16.418642 1 recorder.go:75] Recording config/running_containers with fingerprint=35c7f22fb214c1059265b9a3b982c8a918b3594e34e79d3b1f152732736e3150 I0415 22:54:16.418651 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 61.368174ms to process 2 records I0415 22:54:16.418661 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 62.158778ms to process 0 records I0415 22:54:16.418668 1 tasks_processing.go:74] worker 34 stopped. I0415 22:54:16.420095 1 tasks_processing.go:74] worker 33 stopped. E0415 22:54:16.420107 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0415 22:54:16.420112 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0415 22:54:16.420116 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0415 22:54:16.420124 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0415 22:54:16.420147 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0415 22:54:16.420154 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0415 22:54:16.420170 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0415 22:54:16.420180 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0415 22:54:16.420217 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0415 22:54:16.420224 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0415 22:54:16.420229 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 66.890177ms to process 7 records I0415 22:54:16.422976 1 tasks_processing.go:74] worker 55 stopped. E0415 22:54:16.422987 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0415 22:54:16.422993 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pms8lghlv5kfib42mcplihkudoekrnc-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pms8lghlv5kfib42mcplihkudoekrnc-primary-cert-bundle-secret" not found I0415 22:54:16.423072 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=b082c2b1836db1a5fcb96d67c8613017a4997eb66f790cba672296e45970ae0a I0415 22:54:16.423083 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 69.782334ms to process 1 records I0415 22:54:16.428211 1 base_controller.go:82] Caches are synced for ConfigController I0415 22:54:16.428220 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0415 22:54:16.433609 1 configmapobserver.go:84] configmaps "insights-config" not found I0415 22:54:16.475208 1 tasks_processing.go:74] worker 22 stopped. I0415 22:54:16.475231 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0415 22:54:16.475241 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 121.178916ms to process 1 records I0415 22:54:16.477819 1 cluster_transfer.go:102] no available accepted cluster transfer I0415 22:54:16.477832 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0415 22:54:16.481925 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0415 22:54:16.495379 1 base_controller.go:82] Caches are synced for LoggingSyncer I0415 22:54:16.495390 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0415 22:54:16.727086 1 sca.go:266] etc-pki-entitlement secret successfully created I0415 22:54:16.727198 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated W0415 22:54:17.387571 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0415 22:54:17.416365 1 tasks_processing.go:74] worker 9 stopped. I0415 22:54:17.416385 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0415 22:54:17.416396 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.063239766s to process 1 records I0415 22:54:17.416640 1 tasks_processing.go:74] worker 51 stopped. I0415 22:54:17.416677 1 recorder.go:75] Recording config/alerts with fingerprint=e9ed608323a2dd7b9db57c5170fd7a51628eb4aef0f3955c74bf78c018732a8d I0415 22:54:17.416689 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.063150916s to process 1 records W0415 22:54:18.387542 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0415 22:54:18.818621 1 tasks_processing.go:74] worker 24 stopped. I0415 22:54:18.818693 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=f435b1d2d0bb60c289b39750a6db8853e4e6d93742690183f950039c9ac7b748 I0415 22:54:18.818749 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0415 22:54:18.818773 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=e17903c397aa5da605be62760598a6e876a9deafb145c28a2fac7dbeff460c78 I0415 22:54:18.818783 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0415 22:54:18.818804 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=20d14624edcab3383e057d89c0cad88983b7e825a00b6d8be8842288e5a0223f I0415 22:54:18.818820 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0415 22:54:18.818841 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=e0b70c38560bcac3180b70d1c143cae2ae475b48593cc856974092550a38ebeb I0415 22:54:18.818871 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=9e4d1a2b33dd3b954c56d9641954cef007bbac9452c75255ea9f793e36323475 I0415 22:54:18.818896 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=64282b63760077edd6426aace3b39acdd41145d82955b677262e73baf90077ff I0415 22:54:18.818932 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=4d16384a8c71325561fa0338e6803df8864693515a821aeaefa4945ed8ed4417 I0415 22:54:18.819212 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=1f62335e315a8fceb43eec07e05184bdff33de620f4634b8f255178b291e8cd2 I0415 22:54:18.819232 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=472daba738132f14068831997859e5a4c8ec076c8ccca48516c2d37cb998610a I0415 22:54:18.819241 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0415 22:54:18.819259 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=3ff6093f1a81c67502db762b34103a19fa0f5b5308cb8658c25dd7564fc8c984 I0415 22:54:18.819270 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0415 22:54:18.819285 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=183661bd602995cbbe55d5e186d2874ace0d5ba0ee33733bb6df906b4926ec80 I0415 22:54:18.819293 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0415 22:54:18.819314 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=837175cb071a51f611cb0dd2f6e09790058dd17aa5b4bfc86561ec9a6f250de9 I0415 22:54:18.819324 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0415 22:54:18.819343 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=549f9a22ab060a901d6ed52bcca7ffaaa735f0d8ee939c9ffc167c55852c58d7 I0415 22:54:18.819445 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=baaea6ebf6a8bee161e49b27acce344660dbae502a4498a05a9e9c9afea2faf7 I0415 22:54:18.819455 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0415 22:54:18.819462 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0415 22:54:18.819483 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0415 22:54:18.819505 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=246dd78e55d35cd2dce1625f566df7754c093523c415baad89eb307474ae1de6 I0415 22:54:18.819528 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=566b047f17746a2d597f7a9bd9350eaf1076ea660d3a0e48a07e78e9bcc42023 I0415 22:54:18.819538 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0415 22:54:18.819554 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=ab1801e0a303b88286f6835cb95353fd9bc2b594b373cb26d33580d0bf9dc7e0 I0415 22:54:18.819564 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0415 22:54:18.819581 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=48235f89f98b8174515384e1177612856e9a8b6e5132e958b59077a0fc701fc6 I0415 22:54:18.819591 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0415 22:54:18.819605 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=d41bff5f2fb9e828ca11bccca5918f31753b437d1e8bd5512d80b862b1459593 I0415 22:54:18.819621 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=7671a5c90eebe4c65b306f0d6a980f035ed1e773bc1439fb0cd67b717022b227 I0415 22:54:18.819634 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=7aec8a2196bcd7e5773a5f68448a0bcd5f12434235fdee9d13f09bd6dd38bc77 I0415 22:54:18.819656 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=1dc0cdb47f70982356970156f8f165bab822c59dd5c725903793afd393579dbb I0415 22:54:18.819666 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0415 22:54:18.819693 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=f7d1b5e391c48b59be823f0cabdf2a3fe207af745f5573717fb4935b186d4a98 I0415 22:54:18.819711 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0415 22:54:18.819720 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0415 22:54:18.819727 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.464719086s to process 39 records W0415 22:54:19.386997 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0415 22:54:20.386695 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0415 22:54:21.385422 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0415 22:54:21.385447 1 tasks_processing.go:74] worker 31 stopped. E0415 22:54:21.385458 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0415 22:54:21.385469 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0415 22:54:21.385482 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.032291287s to process 1 records I0415 22:54:29.196385 1 tasks_processing.go:74] worker 19 stopped. I0415 22:54:29.196432 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0415 22:54:29.196445 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.842761403s to process 1 records I0415 22:54:29.960222 1 tasks_processing.go:74] worker 62 stopped. I0415 22:54:29.960501 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=4aff5860ec073ddfdf1bec96303c96ef7753a0d476f8315de0a2062bde3a2fd5 I0415 22:54:29.960516 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.607067948s to process 1 records E0415 22:54:29.960588 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.607s with: function \"machine_healthchecks\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0415 22:54:29.961689 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0415 22:54:29.961704 1 periodic.go:209] Running workloads gatherer I0415 22:54:29.961718 1 tasks_processing.go:45] number of workers: 2 I0415 22:54:29.961730 1 tasks_processing.go:69] worker 1 listening for tasks. I0415 22:54:29.961734 1 tasks_processing.go:71] worker 1 working on workload_info task. I0415 22:54:29.961746 1 tasks_processing.go:69] worker 0 listening for tasks. I0415 22:54:29.961761 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0415 22:54:29.968877 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-226.ec2.internal... I0415 22:54:29.968880 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-197.ec2.internal... I0415 22:54:29.968891 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-214.ec2.internal... I0415 22:54:29.991391 1 tasks_processing.go:74] worker 0 stopped. I0415 22:54:29.991409 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 29.617192ms to process 0 records I0415 22:54:34.381731 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0415 22:54:34.407944 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0415 22:54:34.417753 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (11ms) I0415 22:54:34.428056 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0415 22:54:34.438222 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0415 22:54:34.447471 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (9ms) I0415 22:54:34.457828 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (10ms) I0415 22:54:34.466620 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (9ms) I0415 22:54:34.475479 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (9ms) I0415 22:54:34.484576 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (9ms) I0415 22:54:34.493975 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (9ms) I0415 22:54:34.503458 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (9ms) I0415 22:54:34.518248 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (15ms) I0415 22:54:34.618071 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0415 22:54:34.720512 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (102ms) I0415 22:54:34.818142 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (98ms) I0415 22:54:34.917200 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (99ms) I0415 22:54:35.018613 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (101ms) I0415 22:54:35.121704 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (103ms) I0415 22:54:35.217509 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (96ms) I0415 22:54:35.317329 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (100ms) I0415 22:54:35.417811 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0415 22:54:35.517512 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0415 22:54:35.617416 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0415 22:54:35.717431 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (100ms) I0415 22:54:35.817761 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0415 22:54:35.917669 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0415 22:54:36.027090 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (109ms) I0415 22:54:36.117086 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (90ms) I0415 22:54:36.217775 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0415 22:54:36.317772 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (100ms) I0415 22:54:36.417319 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0415 22:54:36.518805 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (101ms) I0415 22:54:36.620233 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (101ms) I0415 22:54:36.717243 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (97ms) I0415 22:54:36.817668 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (100ms) I0415 22:54:36.917862 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0415 22:54:37.016805 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (99ms) I0415 22:54:37.123366 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (107ms) I0415 22:54:37.217319 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (94ms) I0415 22:54:37.318796 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (101ms) I0415 22:54:37.416992 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (98ms) I0415 22:54:37.517077 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0415 22:54:37.617484 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (100ms) I0415 22:54:37.718815 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0415 22:54:37.817040 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (98ms) I0415 22:54:37.917394 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0415 22:54:38.017613 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0415 22:54:38.117493 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (100ms) I0415 22:54:38.224268 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (107ms) I0415 22:54:38.224296 1 tasks_processing.go:74] worker 1 stopped. I0415 22:54:38.224610 1 recorder.go:75] Recording config/workload_info with fingerprint=c9b93158f93d7b991c2fbe3c3222c4016d9799dc72badece9fb1633ece236c23 I0415 22:54:38.224631 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.262554612s to process 1 records I0415 22:54:38.224648 1 periodic.go:256] Periodic gather workloads completed in 8.262s I0415 22:54:38.224660 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0415 22:54:38.224664 1 periodic.go:209] Running conditional gatherer I0415 22:54:38.231182 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0415 22:54:38.390590 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0415 22:54:39.447584 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0415 22:54:39.447600 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0415 22:54:39.447603 1 conditional_gatherer.go:374] alert "KubeStatefulSetReplicasMismatch" has state "pending" I0415 22:54:39.447606 1 conditional_gatherer.go:374] alert "KubeDaemonSetRolloutStuck" has state "pending" I0415 22:54:39.447609 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0415 22:54:39.447611 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0415 22:54:39.447613 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0415 22:54:39.447616 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0415 22:54:39.447618 1 conditional_gatherer.go:374] alert "KubePdbNotEnoughHealthyPods" has state "pending" I0415 22:54:39.447620 1 conditional_gatherer.go:374] alert "Watchdog" has state "firing" I0415 22:54:39.447692 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0415 22:54:39.455058 1 conditional_gatherer.go:392] cluster version is '4.20.8' I0415 22:54:39.455086 1 tasks_processing.go:45] number of workers: 3 I0415 22:54:39.455098 1 tasks_processing.go:69] worker 2 listening for tasks. I0415 22:54:39.455102 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0415 22:54:39.455109 1 tasks_processing.go:69] worker 0 listening for tasks. I0415 22:54:39.455119 1 tasks_processing.go:69] worker 1 listening for tasks. I0415 22:54:39.455129 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0415 22:54:39.455131 1 tasks_processing.go:74] worker 1 stopped. I0415 22:54:39.455145 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0415 22:54:39.455178 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0415 22:54:39.455190 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 986ns to process 1 records I0415 22:54:39.455223 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0415 22:54:39.455231 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.026µs to process 1 records I0415 22:54:39.455236 1 tasks_processing.go:74] worker 0 stopped. I0415 22:54:39.455412 1 tasks_processing.go:74] worker 2 stopped. I0415 22:54:39.455435 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 252.054µs to process 0 records I0415 22:54:39.455465 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0415 22:54:39.455484 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0415 22:54:39.490466 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=a3c428ada6fdb16c5a70088c9eba62a5a13d72da70d792e96c583e52a0b29320 I0415 22:54:39.490587 1 diskrecorder.go:70] Writing 111 records to /var/lib/insights-operator/insights-2026-04-15-225439.tar.gz I0415 22:54:39.497243 1 diskrecorder.go:51] Wrote 111 records to disk in 6ms I0415 22:54:39.497267 1 periodic.go:278] Gathering cluster info every 2h0m0s I0415 22:54:39.497281 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0415 22:55:46.354024 1 diskrecorder.go:223] Found files to send: insights-2026-04-15-225439.tar.gz I0415 22:55:46.354050 1 insightsuploader.go:150] Checking archives to upload periodically every 16m5.48780252s I0415 22:55:46.354059 1 insightsuploader.go:165] Uploading latest report since 2026-04-15T22:54:11Z I0415 22:55:46.364543 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0415 22:55:46.981085 1 requests.go:87] Successfully reported id=2026-04-15T22:55:46Z x-rh-insights-request-id=42f13c343b00431fbbf54b4ef92c082d, wrote=48589 I0415 22:55:46.981214 1 insightsuploader.go:187] Uploaded report successfully in 627.144961ms I0415 22:55:46.981256 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0415 22:55:46.981272 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0415 22:55:46.981282 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0415 22:55:46.987058 1 controller.go:512] The operator is healthy I0415 22:56:16.402893 1 controller.go:512] The operator is healthy I0415 22:56:50.454991 1 insightsreport.go:137] Pulling report from smart-proxy I0415 22:56:50.455026 1 insightsreport.go:149] Retrieving report I0415 22:56:50.462465 1 requests.go:110] Retrieving report for cluster: e2b27ad5-6eff-4aad-8113-3d368634dc55 I0415 22:56:50.462476 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/e2b27ad5-6eff-4aad-8113-3d368634dc55/reports I0415 22:56:50.465264 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/e2b27ad5-6eff-4aad-8113-3d368634dc55/reports I0415 22:56:50.727795 1 insightsreport.go:184] Report retrieved I0415 22:56:50.752874 1 insightsreport.go:239] Report retrieved correctly I0415 22:58:16.404931 1 controller.go:512] The operator is healthy I0415 22:59:16.345308 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0415 22:59:16.351618 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 22:59:16.351638 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0415 22:59:16.356130 1 secretconfigobserver.go:119] support secret does not exist I0415 23:00:16.404583 1 controller.go:512] The operator is healthy I0415 23:02:16.400537 1 controller.go:512] The operator is healthy I0415 23:04:16.346475 1 configmapobserver.go:84] configmaps "insights-config" not found I0415 23:04:16.356555 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0415 23:04:16.361188 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 23:04:16.361206 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0415 23:04:16.365362 1 secretconfigobserver.go:119] support secret does not exist I0415 23:04:16.399653 1 controller.go:512] The operator is healthy I0415 23:06:16.403833 1 controller.go:512] The operator is healthy I0415 23:08:16.404166 1 controller.go:512] The operator is healthy I0415 23:09:16.366096 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0415 23:09:16.371935 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 23:09:16.371954 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0415 23:09:16.376372 1 secretconfigobserver.go:119] support secret does not exist I0415 23:10:16.401390 1 controller.go:512] The operator is healthy I0415 23:11:57.470789 1 insightsuploader.go:146] Nothing to report since 2026-04-15T22:55:46Z I0415 23:12:16.400761 1 controller.go:512] The operator is healthy I0415 23:14:16.344626 1 configmapobserver.go:84] configmaps "insights-config" not found I0415 23:14:16.353570 1 configmapobserver.go:84] configmaps "insights-config" not found I0415 23:14:16.376645 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0415 23:14:16.382555 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 23:14:16.382574 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0415 23:14:16.389562 1 secretconfigobserver.go:119] support secret does not exist I0415 23:14:16.405709 1 controller.go:512] The operator is healthy I0415 23:16:16.401091 1 controller.go:512] The operator is healthy I0415 23:18:16.402178 1 controller.go:512] The operator is healthy I0415 23:19:16.389871 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0415 23:19:16.396502 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0415 23:19:16.396523 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0415 23:19:16.400802 1 secretconfigobserver.go:119] support secret does not exist I0415 23:20:16.403879 1 controller.go:512] The operator is healthy