I0506 03:16:01.153477 1 cmd.go:253] Using service-serving-cert provided certificates I0506 03:16:01.153936 1 observer_polling.go:159] Starting file observer I0506 03:16:01.776403 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0506 03:16:01.776639 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0506 03:16:01.777137 1 secure_serving.go:57] Forcing use of http/1.1 only W0506 03:16:01.777155 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0506 03:16:01.777160 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0506 03:16:01.777164 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0506 03:16:01.777166 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0506 03:16:01.777169 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0506 03:16:01.777171 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0506 03:16:01.777221 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0506 03:16:01.781546 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0506 03:16:01.781631 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"0182644b-6225-4e61-a822-83da37cf47ef", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0506 03:16:01.784085 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0506 03:16:01.784101 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0506 03:16:01.784116 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0506 03:16:01.784107 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0506 03:16:01.784101 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0506 03:16:01.784162 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0506 03:16:01.784368 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0506 03:16:01.784735 1 secure_serving.go:213] Serving securely on [::]:8443 I0506 03:16:01.784777 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0506 03:16:01.789898 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0506 03:16:01.789919 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0506 03:16:01.790015 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0506 03:16:01.796972 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 03:16:01.796994 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0506 03:16:01.804419 1 secretconfigobserver.go:119] support secret does not exist I0506 03:16:01.810903 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 03:16:01.816196 1 secretconfigobserver.go:119] support secret does not exist I0506 03:16:01.819184 1 recorder.go:161] Pruning old reports every 6h58m12s, max age is 288h0m0s I0506 03:16:01.825875 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0506 03:16:01.825889 1 periodic.go:209] Running clusterconfig gatherer I0506 03:16:01.825893 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0506 03:16:01.825893 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0506 03:16:01.825907 1 insightsreport.go:296] Starting report retriever I0506 03:16:01.825915 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0506 03:16:01.825943 1 tasks_processing.go:45] number of workers: 64 I0506 03:16:01.825979 1 tasks_processing.go:69] worker 3 listening for tasks. I0506 03:16:01.825986 1 tasks_processing.go:69] worker 10 listening for tasks. I0506 03:16:01.825992 1 tasks_processing.go:69] worker 2 listening for tasks. I0506 03:16:01.825994 1 tasks_processing.go:69] worker 4 listening for tasks. I0506 03:16:01.826000 1 tasks_processing.go:69] worker 27 listening for tasks. I0506 03:16:01.826004 1 tasks_processing.go:69] worker 5 listening for tasks. I0506 03:16:01.826003 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 03:16:01.826010 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 03:16:01.826014 1 tasks_processing.go:69] worker 7 listening for tasks. I0506 03:16:01.826015 1 tasks_processing.go:69] worker 12 listening for tasks. I0506 03:16:01.826018 1 tasks_processing.go:71] worker 1 working on certificate_signing_requests task. I0506 03:16:01.826008 1 tasks_processing.go:69] worker 11 listening for tasks. I0506 03:16:01.826022 1 tasks_processing.go:69] worker 9 listening for tasks. I0506 03:16:01.826027 1 tasks_processing.go:69] worker 13 listening for tasks. I0506 03:16:01.826028 1 tasks_processing.go:71] worker 7 working on monitoring_persistent_volumes task. I0506 03:16:01.826033 1 tasks_processing.go:69] worker 14 listening for tasks. I0506 03:16:01.826033 1 tasks_processing.go:71] worker 13 working on install_plans task. I0506 03:16:01.826038 1 tasks_processing.go:71] worker 14 working on storage_classes task. I0506 03:16:01.826036 1 tasks_processing.go:69] worker 8 listening for tasks. I0506 03:16:01.826045 1 tasks_processing.go:69] worker 25 listening for tasks. I0506 03:16:01.826024 1 tasks_processing.go:71] worker 12 working on pdbs task. I0506 03:16:01.826051 1 tasks_processing.go:69] worker 24 listening for tasks. I0506 03:16:01.826028 1 tasks_processing.go:71] worker 11 working on sap_pods task. I0506 03:16:01.826064 1 tasks_processing.go:69] worker 21 listening for tasks. I0506 03:16:01.826068 1 tasks_processing.go:69] worker 16 listening for tasks. I0506 03:16:01.826074 1 tasks_processing.go:69] worker 22 listening for tasks. I0506 03:16:01.826057 1 tasks_processing.go:69] worker 20 listening for tasks. I0506 03:16:01.826085 1 tasks_processing.go:69] worker 37 listening for tasks. I0506 03:16:01.826080 1 tasks_processing.go:69] worker 48 listening for tasks. I0506 03:16:01.826092 1 tasks_processing.go:69] worker 59 listening for tasks. I0506 03:16:01.826095 1 tasks_processing.go:69] worker 38 listening for tasks. I0506 03:16:01.826094 1 tasks_processing.go:69] worker 15 listening for tasks. I0506 03:16:01.826099 1 tasks_processing.go:69] worker 28 listening for tasks. I0506 03:16:01.826101 1 tasks_processing.go:69] worker 49 listening for tasks. I0506 03:16:01.826110 1 tasks_processing.go:69] worker 50 listening for tasks. I0506 03:16:01.826107 1 tasks_processing.go:69] worker 39 listening for tasks. I0506 03:16:01.826107 1 tasks_processing.go:69] worker 29 listening for tasks. I0506 03:16:01.826113 1 tasks_processing.go:69] worker 40 listening for tasks. I0506 03:16:01.826118 1 tasks_processing.go:69] worker 30 listening for tasks. I0506 03:16:01.826115 1 tasks_processing.go:69] worker 26 listening for tasks. I0506 03:16:01.826125 1 tasks_processing.go:69] worker 17 listening for tasks. I0506 03:16:01.826119 1 tasks_processing.go:69] worker 51 listening for tasks. I0506 03:16:01.826120 1 tasks_processing.go:69] worker 41 listening for tasks. I0506 03:16:01.826133 1 tasks_processing.go:69] worker 32 listening for tasks. I0506 03:16:01.826126 1 tasks_processing.go:69] worker 31 listening for tasks. I0506 03:16:01.826127 1 tasks_processing.go:69] worker 42 listening for tasks. I0506 03:16:01.826128 1 tasks_processing.go:69] worker 52 listening for tasks. I0506 03:16:01.826142 1 tasks_processing.go:69] worker 33 listening for tasks. I0506 03:16:01.826135 1 tasks_processing.go:69] worker 55 listening for tasks. I0506 03:16:01.826143 1 tasks_processing.go:69] worker 54 listening for tasks. I0506 03:16:01.826135 1 tasks_processing.go:69] worker 43 listening for tasks. I0506 03:16:01.826150 1 tasks_processing.go:69] worker 45 listening for tasks. I0506 03:16:01.826135 1 tasks_processing.go:69] worker 56 listening for tasks. I0506 03:16:01.826157 1 tasks_processing.go:69] worker 46 listening for tasks. I0506 03:16:01.826158 1 tasks_processing.go:69] worker 36 listening for tasks. I0506 03:16:01.826159 1 tasks_processing.go:69] worker 47 listening for tasks. I0506 03:16:01.826164 1 tasks_processing.go:71] worker 4 working on silenced_alerts task. I0506 03:16:01.826166 1 tasks_processing.go:71] worker 10 working on version task. I0506 03:16:01.826158 1 tasks_processing.go:69] worker 53 listening for tasks. I0506 03:16:01.826171 1 tasks_processing.go:69] worker 61 listening for tasks. I0506 03:16:01.826140 1 tasks_processing.go:71] worker 9 working on cost_management_metrics_configs task. I0506 03:16:01.826174 1 tasks_processing.go:71] worker 5 working on config_maps task. I0506 03:16:01.826180 1 tasks_processing.go:69] worker 60 listening for tasks. I0506 03:16:01.826184 1 tasks_processing.go:69] worker 62 listening for tasks. I0506 03:16:01.826149 1 tasks_processing.go:69] worker 58 listening for tasks. I0506 03:16:01.826175 1 tasks_processing.go:69] worker 35 listening for tasks. I0506 03:16:01.826142 1 tasks_processing.go:69] worker 44 listening for tasks. I0506 03:16:01.826164 1 tasks_processing.go:69] worker 18 listening for tasks. I0506 03:16:01.826169 1 tasks_processing.go:71] worker 2 working on validating_webhook_configurations task. I0506 03:16:01.826199 1 tasks_processing.go:69] worker 63 listening for tasks. I0506 03:16:01.826142 1 tasks_processing.go:69] worker 57 listening for tasks. I0506 03:16:01.826184 1 tasks_processing.go:71] worker 27 working on machine_healthchecks task. I0506 03:16:01.826017 1 tasks_processing.go:71] worker 0 working on mutating_webhook_configurations task. I0506 03:16:01.826009 1 tasks_processing.go:69] worker 6 listening for tasks. I0506 03:16:01.826268 1 tasks_processing.go:71] worker 55 working on infrastructures task. I0506 03:16:01.826268 1 tasks_processing.go:71] worker 30 working on image task. I0506 03:16:01.826269 1 tasks_processing.go:71] worker 43 working on nodenetworkstates task. I0506 03:16:01.826268 1 tasks_processing.go:71] worker 46 working on ingress_certificates task. I0506 03:16:01.826271 1 tasks_processing.go:71] worker 57 working on crds task. I0506 03:16:01.826270 1 tasks_processing.go:71] worker 45 working on sap_config task. I0506 03:16:01.826273 1 tasks_processing.go:71] worker 8 working on sap_datahubs task. I0506 03:16:01.826272 1 tasks_processing.go:71] worker 26 working on jaegers task. I0506 03:16:01.826273 1 tasks_processing.go:71] worker 54 working on aggregated_monitoring_cr_names task. I0506 03:16:01.826275 1 tasks_processing.go:71] worker 32 working on qemu_kubevirt_launcher_logs task. I0506 03:16:01.826274 1 tasks_processing.go:71] worker 56 working on openshift_machine_api_events task. I0506 03:16:01.826277 1 tasks_processing.go:71] worker 25 working on pod_network_connectivity_checks task. I0506 03:16:01.826277 1 tasks_processing.go:71] worker 62 working on metrics task. I0506 03:16:01.826278 1 tasks_processing.go:71] worker 31 working on machine_autoscalers task. I0506 03:16:01.826276 1 tasks_processing.go:71] worker 36 working on proxies task. I0506 03:16:01.826280 1 tasks_processing.go:71] worker 44 working on service_accounts task. I0506 03:16:01.826281 1 tasks_processing.go:71] worker 17 working on schedulers task. I0506 03:16:01.826281 1 tasks_processing.go:71] worker 38 working on cluster_apiserver task. I0506 03:16:01.826282 1 tasks_processing.go:71] worker 47 working on openstack_dataplanedeployments task. I0506 03:16:01.826282 1 tasks_processing.go:71] worker 59 working on openstack_version task. I0506 03:16:01.826283 1 tasks_processing.go:71] worker 42 working on overlapping_namespace_uids task. I0506 03:16:01.826284 1 tasks_processing.go:71] worker 18 working on container_runtime_configs task. I0506 03:16:01.826284 1 tasks_processing.go:71] worker 58 working on nodes task. I0506 03:16:01.826286 1 tasks_processing.go:71] worker 15 working on machine_sets task. I0506 03:16:01.826286 1 tasks_processing.go:71] worker 53 working on nodenetworkconfigurationpolicies task. I0506 03:16:01.826287 1 tasks_processing.go:71] worker 52 working on machine_config_pools task. I0506 03:16:01.826287 1 tasks_processing.go:71] worker 24 working on ceph_cluster task. I0506 03:16:01.826285 1 tasks_processing.go:71] worker 51 working on feature_gates task. I0506 03:16:01.826288 1 tasks_processing.go:71] worker 63 working on ingress task. I0506 03:16:01.826291 1 tasks_processing.go:71] worker 61 working on openstack_dataplanenodesets task. I0506 03:16:01.826291 1 tasks_processing.go:71] worker 21 working on networks task. I0506 03:16:01.826288 1 tasks_processing.go:71] worker 35 working on tsdb_status task. I0506 03:16:01.826158 1 tasks_processing.go:71] worker 3 working on active_alerts task. I0506 03:16:01.826290 1 tasks_processing.go:71] worker 33 working on oauths task. I0506 03:16:01.826292 1 tasks_processing.go:71] worker 41 working on openstack_controlplanes task. I0506 03:16:01.826295 1 tasks_processing.go:71] worker 20 working on storage_cluster task. I0506 03:16:01.826295 1 tasks_processing.go:71] worker 37 working on lokistack task. I0506 03:16:01.826295 1 tasks_processing.go:71] worker 60 working on olm_operators task. I0506 03:16:01.826296 1 tasks_processing.go:71] worker 16 working on node_logs task. I0506 03:16:01.826298 1 tasks_processing.go:71] worker 48 working on image_pruners task. I0506 03:16:01.826298 1 tasks_processing.go:71] worker 39 working on clusterroles task. I0506 03:16:01.826299 1 tasks_processing.go:71] worker 22 working on machines task. I0506 03:16:01.826301 1 tasks_processing.go:71] worker 49 working on authentication task. I0506 03:16:01.826302 1 tasks_processing.go:71] worker 50 working on dvo_metrics task. I0506 03:16:01.826303 1 tasks_processing.go:71] worker 29 working on machine_configs task. I0506 03:16:01.826289 1 tasks_processing.go:71] worker 28 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0506 03:16:01.826305 1 tasks_processing.go:71] worker 40 working on support_secret task. I0506 03:16:01.826045 1 tasks_processing.go:69] worker 23 listening for tasks. I0506 03:16:01.829341 1 tasks_processing.go:71] worker 23 working on image_registries task. I0506 03:16:01.826021 1 tasks_processing.go:69] worker 19 listening for tasks. I0506 03:16:01.829400 1 tasks_processing.go:71] worker 19 working on openshift_logging task. I0506 03:16:01.826151 1 tasks_processing.go:69] worker 34 listening for tasks. I0506 03:16:01.829482 1 tasks_processing.go:71] worker 34 working on operators_pods_and_events task. I0506 03:16:01.826382 1 tasks_processing.go:71] worker 6 working on operators task. E0506 03:16:01.831600 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.43.79:9094: connect: connection refused I0506 03:16:01.831617 1 tasks_processing.go:71] worker 3 working on container_images task. E0506 03:16:01.831638 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.43.79:9094: connect: connection refused E0506 03:16:01.831662 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.236.35:9091: connect: connection refused E0506 03:16:01.831663 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.43.79:9094: connect: connection refused E0506 03:16:01.831668 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.236.35:9091: connect: connection refused I0506 03:16:01.831680 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 3.237389ms to process 0 records E0506 03:16:01.831693 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.43.79:9094: connect: connection refused I0506 03:16:01.831702 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 5.486138ms to process 0 records E0506 03:16:01.831710 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.236.35:9091: connect: connection refused I0506 03:16:01.831715 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 3.306894ms to process 0 records I0506 03:16:01.831726 1 tasks_processing.go:74] worker 4 stopped. I0506 03:16:01.831731 1 tasks_processing.go:74] worker 35 stopped. I0506 03:16:01.831727 1 tasks_processing.go:74] worker 62 stopped. E0506 03:16:01.831722 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.236.35:9091: connect: connection refused I0506 03:16:01.832046 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 4.348879ms to process 0 records I0506 03:16:01.833485 1 tasks_processing.go:74] worker 11 stopped. I0506 03:16:01.833502 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 7.417832ms to process 0 records I0506 03:16:01.833767 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0506 03:16:01.833784 1 controller.go:326] The initial operator extension status is healthy I0506 03:16:01.833791 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0506 03:16:01.833796 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0506 03:16:01.833799 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0506 03:16:01.833812 1 controller.go:489] The operator is still being initialized I0506 03:16:01.833818 1 controller.go:512] The operator is healthy I0506 03:16:01.833891 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0506 03:16:01.833959 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0506 03:16:01.834011 1 operator.go:288] started I0506 03:16:01.834033 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0506 03:16:01.834380 1 tasks_processing.go:74] worker 9 stopped. I0506 03:16:01.834392 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 8.195587ms to process 0 records I0506 03:16:01.844765 1 tasks_processing.go:74] worker 24 stopped. I0506 03:16:01.844777 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 16.742616ms to process 0 records E0506 03:16:01.844785 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0506 03:16:01.844791 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 17.544426ms to process 0 records I0506 03:16:01.844802 1 tasks_processing.go:74] worker 25 stopped. I0506 03:16:01.844913 1 tasks_processing.go:74] worker 12 stopped. I0506 03:16:01.844928 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=b153fc492ce98c74acab9b63ef1d1bee6093ae6f8df24b5b8494c8698356eb1b I0506 03:16:01.844945 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=d3b5b86f903c3f6749f040a8b9d544ab0dc7f771a614ccd499e9f42c4451e772 I0506 03:16:01.844958 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=28ff7062300a5f50d2398f4dd4361242b11ec59928c490864c71a8a2517398bd I0506 03:16:01.844979 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=fc50e188b55266bd132000572be5fdd76367df71712f1e529e00324c021fff07 I0506 03:16:01.845004 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=64ac54cebca0ff0b4b450d7ff42ca9d6c83dc03d3b7129585b4b9bc2a00a6f29 I0506 03:16:01.845027 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=d6aba90ef0c7a646cb162b242c268bea11bd09b01a96b264e18297f2eab362a7 I0506 03:16:01.845050 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=a36d55244eb99be97fd07eb3dd90c2a2d777797b573892f24177c87a619329bc I0506 03:16:01.845074 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=146901427a398d4eecd51de68de1646b39f6dc57743edd35861ec768d0807835 I0506 03:16:01.845101 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=7c28d2447adc20a97080b3b3e10d26a310735a98d4552de4be3cf62152762299 I0506 03:16:01.845129 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=53012d001cb825ee5fbec5db53b9b2a26decaf3f83c019845d0fcba4f89a4687 I0506 03:16:01.845140 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 18.708825ms to process 10 records I0506 03:16:01.845152 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 17.150644ms to process 0 records I0506 03:16:01.845159 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 18.249238ms to process 0 records I0506 03:16:01.845165 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 17.139993ms to process 0 records E0506 03:16:01.845173 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0506 03:16:01.845173 1 tasks_processing.go:74] worker 47 stopped. I0506 03:16:01.845180 1 tasks_processing.go:74] worker 43 stopped. I0506 03:16:01.845181 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 18.680542ms to process 0 records I0506 03:16:01.845185 1 tasks_processing.go:74] worker 59 stopped. I0506 03:16:01.845190 1 tasks_processing.go:74] worker 27 stopped. E0506 03:16:01.845194 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0506 03:16:01.845198 1 tasks_processing.go:74] worker 22 stopped. I0506 03:16:01.845200 1 gather.go:177] gatherer "clusterconfig" function "machines" took 15.985688ms to process 0 records I0506 03:16:01.845207 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 18.202865ms to process 0 records I0506 03:16:01.845215 1 tasks_processing.go:74] worker 45 stopped. I0506 03:16:01.848604 1 tasks_processing.go:74] worker 19 stopped. I0506 03:16:01.848618 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 19.196299ms to process 0 records I0506 03:16:01.854940 1 tasks_processing.go:74] worker 55 stopped. I0506 03:16:01.855491 1 recorder.go:75] Recording config/infrastructure with fingerprint=7d1a50b24c32838b48911ddd82366c5605e55c35dfc7cc86802b7dfe14627036 I0506 03:16:01.855505 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 28.545867ms to process 1 records I0506 03:16:01.855578 1 recorder.go:75] Recording config/image with fingerprint=67093f5e962488d6c6aed66213887b8eb173125a81dbc18ae542cec2e8cdf020 I0506 03:16:01.855588 1 gather.go:177] gatherer "clusterconfig" function "image" took 28.45106ms to process 1 records I0506 03:16:01.855593 1 tasks_processing.go:74] worker 30 stopped. I0506 03:16:01.855663 1 tasks_processing.go:74] worker 48 stopped. I0506 03:16:01.855794 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=f4e41d814deb4a7888e6bb0de2e01de6f5be6f78ac92d895cdbf40615d6947a3 I0506 03:16:01.855803 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 26.324155ms to process 1 records I0506 03:16:01.856623 1 tasks_processing.go:74] worker 1 stopped. I0506 03:16:01.856641 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 30.588119ms to process 0 records I0506 03:16:01.859491 1 tasks_processing.go:74] worker 37 stopped. I0506 03:16:01.859501 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 30.9193ms to process 0 records I0506 03:16:01.860003 1 tasks_processing.go:74] worker 21 stopped. I0506 03:16:01.860146 1 recorder.go:75] Recording config/network with fingerprint=56be3387bd786467ff4f12b3a30aff9e03cd3f3381bf1751aa9f1bcf5338c5bb I0506 03:16:01.860163 1 gather.go:177] gatherer "clusterconfig" function "networks" took 31.662692ms to process 1 records E0506 03:16:01.860177 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0506 03:16:01.860188 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 30.733852ms to process 0 records I0506 03:16:01.860194 1 tasks_processing.go:74] worker 40 stopped. I0506 03:16:01.860505 1 tasks_processing.go:74] worker 49 stopped. I0506 03:16:01.860659 1 recorder.go:75] Recording config/authentication with fingerprint=a1b5840a7ff4853400691ecc0017b9e2814a2abb9c2414e4e3733fe5e72f2b71 I0506 03:16:01.860674 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 31.466436ms to process 1 records I0506 03:16:01.861190 1 tasks_processing.go:74] worker 8 stopped. I0506 03:16:01.861204 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 34.349731ms to process 0 records I0506 03:16:01.861439 1 tasks_processing.go:74] worker 26 stopped. I0506 03:16:01.861468 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 34.488102ms to process 0 records I0506 03:16:01.861556 1 tasks_processing.go:74] worker 63 stopped. I0506 03:16:01.861662 1 recorder.go:75] Recording config/ingress with fingerprint=d3147d9fc9d2eb3a6802b062473f6452b6da9218d31c8076bba0236f9db18e55 I0506 03:16:01.861684 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 33.293849ms to process 1 records I0506 03:16:01.861779 1 tasks_processing.go:74] worker 38 stopped. I0506 03:16:01.861886 1 recorder.go:75] Recording config/apiserver with fingerprint=7604ca7411c8fdcaf1d923362d2d00270528a335b664e8636c9acff59c7c7c33 I0506 03:16:01.861898 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 34.05819ms to process 1 records I0506 03:16:01.872816 1 tasks_processing.go:74] worker 16 stopped. I0506 03:16:01.872830 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 44.051818ms to process 0 records I0506 03:16:01.873581 1 tasks_processing.go:74] worker 42 stopped. I0506 03:16:01.873604 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0506 03:16:01.873616 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 45.83814ms to process 1 records I0506 03:16:01.873922 1 tasks_processing.go:74] worker 61 stopped. I0506 03:16:01.873933 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 45.670275ms to process 0 records I0506 03:16:01.873946 1 tasks_processing.go:74] worker 15 stopped. I0506 03:16:01.873968 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 46.077678ms to process 0 records I0506 03:16:01.873978 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 46.140114ms to process 0 records I0506 03:16:01.873983 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 46.807905ms to process 0 records I0506 03:16:01.873986 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 45.526749ms to process 0 records I0506 03:16:01.873990 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 45.485492ms to process 0 records I0506 03:16:01.873993 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 46.622942ms to process 0 records I0506 03:16:01.873998 1 tasks_processing.go:74] worker 31 stopped. I0506 03:16:01.874001 1 tasks_processing.go:74] worker 18 stopped. I0506 03:16:01.874005 1 tasks_processing.go:74] worker 56 stopped. I0506 03:16:01.874008 1 tasks_processing.go:74] worker 41 stopped. I0506 03:16:01.874015 1 tasks_processing.go:74] worker 20 stopped. I0506 03:16:01.874072 1 tasks_processing.go:74] worker 58 stopped. I0506 03:16:01.874346 1 recorder.go:75] Recording config/node/ip-10-0-0-203.ec2.internal with fingerprint=8cab3f5c031eb2c719beda5dac2a9aecd34003919156e5adc2075e87e33be821 I0506 03:16:01.874402 1 recorder.go:75] Recording config/node/ip-10-0-2-43.ec2.internal with fingerprint=a59b5a6af54a575f975c07210b15948a472fc735b17d6e7a2001699087f54c02 I0506 03:16:01.874411 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 46.213212ms to process 2 records I0506 03:16:01.874483 1 tasks_processing.go:74] worker 33 stopped. I0506 03:16:01.874670 1 recorder.go:75] Recording config/oauth with fingerprint=a5ffbc8a7dde7023b2da9c806df099e07b4841badf9e76a60f0244c3cc9e6603 I0506 03:16:01.874680 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 45.738272ms to process 1 records I0506 03:16:01.884141 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0506 03:16:01.884164 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0506 03:16:01.884243 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0506 03:16:01.884635 1 tasks_processing.go:74] worker 53 stopped. I0506 03:16:01.884663 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 56.727506ms to process 0 records I0506 03:16:01.885104 1 tasks_processing.go:74] worker 60 stopped. I0506 03:16:01.885150 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 56.38728ms to process 0 records I0506 03:16:01.885324 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=a4cb498d2e0eee0eb3e6546f58900b0f253ab41e2d0c26245f8f4635fad8111c I0506 03:16:01.885361 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 57.593179ms to process 1 records I0506 03:16:01.885390 1 tasks_processing.go:74] worker 17 stopped. I0506 03:16:01.886968 1 tasks_processing.go:74] worker 7 stopped. I0506 03:16:01.886990 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 60.929102ms to process 0 records I0506 03:16:01.887001 1 gather_logs.go:145] no pods in namespace were found I0506 03:16:01.887009 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 59.041633ms to process 0 records I0506 03:16:01.887042 1 tasks_processing.go:74] worker 52 stopped. I0506 03:16:01.887117 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=a1c43be915d6ce8ffe70c6c3572127a446700bad064d240acfa20af1dd645220 I0506 03:16:01.887129 1 tasks_processing.go:74] worker 14 stopped. I0506 03:16:01.887140 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=9eab1055636e74c9e571652dd2aa2628630654669e830cba01eb315c5c2b0f71 I0506 03:16:01.887147 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 60.940748ms to process 2 records I0506 03:16:01.887153 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 59.942586ms to process 0 records I0506 03:16:01.887160 1 tasks_processing.go:74] worker 32 stopped. W0506 03:16:01.887508 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 03:16:01.887707 1 tasks_processing.go:74] worker 23 stopped. I0506 03:16:01.887920 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0506 03:16:01.887931 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0506 03:16:01.887935 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0506 03:16:01.887937 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0506 03:16:01.887941 1 controller.go:212] Source scaController *sca.Controller is not ready I0506 03:16:01.887957 1 controller.go:489] The operator is still being initialized I0506 03:16:01.887963 1 controller.go:512] The operator is healthy I0506 03:16:01.888217 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=de1611fe0efd640be410388b2f32746349db4f51facd1c2f75479155114713b3 I0506 03:16:01.888234 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 58.355415ms to process 1 records I0506 03:16:01.888325 1 tasks_processing.go:74] worker 2 stopped. I0506 03:16:01.888417 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=edf992281483ec7de12a2342b1c2f67605ca092fb41fd2884894fd60a1236e9d I0506 03:16:01.888518 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=14d0414848d6705aa7303aecc671821515c4d9e5b4a3d2b7d3faeab7d2444a4d I0506 03:16:01.888649 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=ac0dc22bbe94ad252d6fab54691f5c68912d8ca291469ab3bcc504ef2f858493 I0506 03:16:01.888688 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=0128c6ca40b4e573490c4c314d152dfc250be915560dad9e25a81848598c6e3d I0506 03:16:01.888734 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=7d87aa5092533ce0b2b6a5cc4e92750aa7dd9bbcff5603783fb6256a248e5f02 I0506 03:16:01.888774 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=10c49d193ddfbdea1277d42dd97948d50d4534de469dcfaeab1969937dc7a48b I0506 03:16:01.888813 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=f366665dd0e708cf2be704db86e7c5b31700aa2475403fd721403bd74890758e I0506 03:16:01.888853 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=481bb33e6b30f0d44404889b1e20cb1e1491ce1b63f0c01fe770ca1926e62b2f I0506 03:16:01.888951 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=3f5a7610632024d06a38d7eb52531c06a7c05720f0cf22d327026697559e59cb I0506 03:16:01.889024 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=cdd708bf812ef9ce4ab214817197247bfc83703a10b138280666d46c7de34d38 I0506 03:16:01.889063 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=488a6b337e4930f0f3553ee9aff62039a38be7d2446b26121e3647a48f161d2a I0506 03:16:01.889119 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=be82b949408bf998b07f13401944b663d3bdab371ebec75cdc1546c5c575f642 I0506 03:16:01.889161 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=b82d0a64f9553e9ff897d01963b9ef2e117a54fa889696e789658ba31f9f59a1 I0506 03:16:01.889173 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 61.862421ms to process 13 records I0506 03:16:01.889896 1 tasks_processing.go:74] worker 34 stopped. I0506 03:16:01.889909 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 60.401614ms to process 0 records I0506 03:16:01.890076 1 base_controller.go:82] Caches are synced for ConfigController I0506 03:16:01.890087 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0506 03:16:01.896645 1 tasks_processing.go:74] worker 51 stopped. I0506 03:16:01.896864 1 recorder.go:75] Recording config/featuregate with fingerprint=7d2b0bd0f0358cfb15f755a916adafddf42c5ce2fc5ba0650aba82d7ddd1052a I0506 03:16:01.896910 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 68.548745ms to process 1 records I0506 03:16:01.897004 1 tasks_processing.go:74] worker 36 stopped. I0506 03:16:01.897026 1 recorder.go:75] Recording config/proxy with fingerprint=d02b8b58391ea02d18d1c3967b862e06f61475a74658898c15244e0f85d4161d I0506 03:16:01.897038 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 69.477998ms to process 1 records I0506 03:16:01.897344 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=b68b7cda992bd08def9e8b7ca4f2bb1e38eb4deb8fa5ff9c808d28f9a06c6694 I0506 03:16:01.897413 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=4f0d1360babf0aefc6dbe840d69f0632cbb9c9c073c1726aacb7f22f36ac94ea I0506 03:16:01.897531 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=674bfe3f71fe720a46b638198046adf9dc3fb9be1813a51a149aa88009bfce3b I0506 03:16:01.897552 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 70.612568ms to process 3 records I0506 03:16:01.897567 1 tasks_processing.go:74] worker 0 stopped. I0506 03:16:01.898154 1 tasks_processing.go:74] worker 57 stopped. I0506 03:16:01.899824 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=a20f53112c8981c6d46ad92faa6e94b9aeaffb4e626bbac65f4f535c2dc93034 I0506 03:16:01.900043 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=fdaeb8cf8b3cfa77edf45c196060605e1439e6145949d95197150aa53031ab0a I0506 03:16:01.900058 1 gather.go:177] gatherer "clusterconfig" function "crds" took 71.54172ms to process 2 records I0506 03:16:01.920715 1 tasks_processing.go:74] worker 39 stopped. I0506 03:16:01.920888 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=a484e40cdb74771f0be87ac363f8b702893b3f46e509c076f7a3aec91106ed98 I0506 03:16:01.920969 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=e66ac5fc8bbd389833a4d244849e6606a123753df9d849665eb63c9a9ea2e0a4 I0506 03:16:01.920977 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 91.804616ms to process 2 records I0506 03:16:01.928679 1 tasks_processing.go:74] worker 10 stopped. I0506 03:16:01.928761 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 03:16:01.928948 1 recorder.go:75] Recording config/version with fingerprint=9b504152114df6e1470337d87bd36c5daa2a491e2163367cb4136d8a63e0b702 I0506 03:16:01.928963 1 recorder.go:75] Recording config/id with fingerprint=375974ff3e5d0af2227e77098768ce1caa4d09a2782d6208fce1e80dbf54da04 I0506 03:16:01.928970 1 gather.go:177] gatherer "clusterconfig" function "version" took 102.498515ms to process 2 records I0506 03:16:01.930603 1 tasks_processing.go:74] worker 54 stopped. I0506 03:16:01.930631 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 103.568043ms to process 0 records I0506 03:16:01.931778 1 tasks_processing.go:74] worker 29 stopped. I0506 03:16:01.931808 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0506 03:16:01.931818 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 102.536709ms to process 1 records I0506 03:16:01.934017 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0506 03:16:01.934079 1 base_controller.go:82] Caches are synced for LoggingSyncer I0506 03:16:01.934093 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0506 03:16:01.937984 1 tasks_processing.go:74] worker 3 stopped. I0506 03:16:01.939228 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-h8q28 with fingerprint=01f223e5bba3043fc784b5817a33e5e78b223dda21f7561f263a82986fd66013 I0506 03:16:01.939351 1 recorder.go:75] Recording config/pod/openshift-multus/multus-8d2n9 with fingerprint=d991ae5fdbb43c1342ef6696b65abd9e830b9cdb739fa8df5bdf06b1e91cb4f4 I0506 03:16:01.939425 1 recorder.go:75] Recording config/running_containers with fingerprint=916fa3945fbb1b8f0f7d016a7a3895a769b35760abf873770a685bf832a3145e I0506 03:16:01.939434 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 106.351894ms to process 3 records I0506 03:16:01.939445 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 108.935308ms to process 0 records I0506 03:16:01.939468 1 tasks_processing.go:74] worker 28 stopped. I0506 03:16:01.943025 1 tasks_processing.go:74] worker 46 stopped. E0506 03:16:01.943043 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0506 03:16:01.943051 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q45sv9i6jjuk0kvdolcn94hju6tqsuq-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q45sv9i6jjuk0kvdolcn94hju6tqsuq-primary-cert-bundle-secret" not found I0506 03:16:01.943117 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=d18689b1e0164a3c6e5ffb4ebe9dd3948bce9cf927fbf7ed55e5876d1e04ca0e I0506 03:16:01.943133 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 116.430544ms to process 1 records I0506 03:16:01.948118 1 tasks_processing.go:74] worker 5 stopped. E0506 03:16:01.948138 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0506 03:16:01.948154 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0506 03:16:01.948159 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0506 03:16:01.948171 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0506 03:16:01.948215 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0506 03:16:01.948225 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0506 03:16:01.948249 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0506 03:16:01.948265 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0506 03:16:01.948327 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0506 03:16:01.948337 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0506 03:16:01.948345 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 121.922896ms to process 7 records I0506 03:16:01.998340 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0506 03:16:02.008637 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0506 03:16:02.020068 1 cluster_transfer.go:102] no available accepted cluster transfer I0506 03:16:02.020084 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0506 03:16:02.152025 1 sca.go:266] etc-pki-entitlement secret successfully created I0506 03:16:02.152045 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated W0506 03:16:02.886854 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0506 03:16:03.886554 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 03:16:04.347535 1 tasks_processing.go:74] worker 6 stopped. I0506 03:16:04.347648 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=244ab03748e0ba5062bf9363ac77b436a56c65ff0177c8f171e81c986780d275 I0506 03:16:04.347694 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0506 03:16:04.347717 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=a1c76f78f1d6e3cd48b4be85b4f42e3ca7c457f60db2933a9e90173f0d490cae I0506 03:16:04.347727 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0506 03:16:04.347749 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=0e6f6f3a10e1638c914fdc05027fb3719469de718dac9f609aaeb554ba561279 I0506 03:16:04.347765 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0506 03:16:04.347788 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=0cde05b5d98f99323b8e39f44aabf4ba0aab2055c9e9baebffec10d9952aaa12 I0506 03:16:04.347815 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=e889deae6908629027331c008e178041aa6337a9123b469298f9064b1dd1f816 I0506 03:16:04.347838 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=ddbd71956164c174efb2629823360025f772491910fb96f3ef3040427ea25b45 I0506 03:16:04.347867 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=513d6bba65ddb7105a8fefd40194d5ca9bb8f1dd549e06a19900fbc82ceab448 I0506 03:16:04.348103 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=de1fda30d7165c1a0f2331aa8b3b3f817e7ac6647ecd7db9b026f6f1203bca9f I0506 03:16:04.348122 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=39d4509435bba0484966defb51b71ffb2f2b2772173068c7b445c4c51e01b35b I0506 03:16:04.348129 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0506 03:16:04.348149 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=43d7ba4031ff324d970e1ac22af7affa1ecefd465149f0bcabf115d07f02a253 I0506 03:16:04.348165 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0506 03:16:04.348186 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=87b6e01cd69f8068e48362923813da8526fbe5486a1567623ce457e1aba23f46 I0506 03:16:04.348196 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0506 03:16:04.348229 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=04f98680f485f7661d5104a03ae6b18e13e0b968a4f3be0e413ea58609f4c5b4 I0506 03:16:04.348244 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0506 03:16:04.348276 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=c3136e38f369e629dc6363547b1936081c19d944df653abf747d4683a99b3b76 I0506 03:16:04.348391 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=c6463213c87cd0b15b5c8d3f8dc33e2a2678438fa30c06498cf9c34688464bdd I0506 03:16:04.348408 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0506 03:16:04.348421 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0506 03:16:04.348485 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0506 03:16:04.348522 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=a7622cff48e01542faea79c12520424c7e0c1862ee677bdba9362232c71721e3 I0506 03:16:04.348561 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=6838ef763b9076ed7a52181529a95a6af12bf046423a5c32d62cfa50faae8003 I0506 03:16:04.348575 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0506 03:16:04.348602 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=ab2348046612cce6e09e8e162a3d9912ebcac4d92e7f36b1b1a62b3e44d1d54c I0506 03:16:04.348618 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0506 03:16:04.348646 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=6c6e4fab62e9f9ff8061b895d00f6d8d7576214ce7a7e0abe79e095b70a48944 I0506 03:16:04.348661 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0506 03:16:04.348682 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=8c5c79e4b44f517201bca14fc4b33811748a574fdd0096679380a71887bd5978 I0506 03:16:04.348707 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=3aa58343f334a2a5d6daa745ea667ee56d79d760c86301f9b0e28529727e6d42 I0506 03:16:04.348733 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=74cd7b178ebc4ce0565902a94ef1f554ccf3e9fb4ecd3ad6849a85b57ba0f2c2 I0506 03:16:04.348765 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=5fd80ba86d14cf76fb83d90efb8d5c43a13e0c67815bf9d46e19faddc0f33990 I0506 03:16:04.348778 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0506 03:16:04.348815 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=5177253a85f67fe5bc77b3c280da5e271f0120c3b4ae3fc9eab3bce0de47b0d3 I0506 03:16:04.348842 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0506 03:16:04.348857 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0506 03:16:04.348869 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.517958824s to process 39 records W0506 03:16:04.885701 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0506 03:16:05.885879 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0506 03:16:06.884856 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0506 03:16:06.884882 1 tasks_processing.go:74] worker 50 stopped. E0506 03:16:06.884894 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0506 03:16:06.884906 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0506 03:16:06.884916 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.055655429s to process 1 records I0506 03:16:14.703003 1 tasks_processing.go:74] worker 13 stopped. I0506 03:16:14.703052 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0506 03:16:14.703067 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.876952324s to process 1 records I0506 03:16:15.439272 1 tasks_processing.go:74] worker 44 stopped. I0506 03:16:15.439565 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=a788ad78cbd67d5a6638eb1140c4d716876f544fdaf6e23c46a9878dbe7a0d0b I0506 03:16:15.439581 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.611813886s to process 1 records E0506 03:16:15.439647 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.613s with: function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0506 03:16:15.440758 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0506 03:16:15.440775 1 periodic.go:209] Running workloads gatherer I0506 03:16:15.440798 1 tasks_processing.go:45] number of workers: 2 I0506 03:16:15.440813 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 03:16:15.440819 1 tasks_processing.go:71] worker 1 working on workload_info task. I0506 03:16:15.440826 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 03:16:15.440840 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0506 03:16:15.450930 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-203.ec2.internal... I0506 03:16:15.450939 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-43.ec2.internal... I0506 03:16:15.469658 1 tasks_processing.go:74] worker 0 stopped. I0506 03:16:15.469672 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 28.805099ms to process 0 records I0506 03:16:21.284337 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 6s I0506 03:16:21.310032 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 23s for image data I0506 03:16:21.320139 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (11ms) I0506 03:16:21.330298 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0506 03:16:21.340893 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (11ms) I0506 03:16:21.353221 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (12ms) I0506 03:16:21.363396 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (10ms) I0506 03:16:21.373308 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (10ms) I0506 03:16:21.384053 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (11ms) I0506 03:16:21.394208 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (10ms) I0506 03:16:21.404621 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (10ms) I0506 03:16:21.414899 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (10ms) I0506 03:16:21.425813 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (11ms) I0506 03:16:21.520701 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (95ms) I0506 03:16:21.620688 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0506 03:16:21.724265 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (104ms) I0506 03:16:21.821002 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (96ms) I0506 03:16:21.920507 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (99ms) I0506 03:16:22.020854 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0506 03:16:22.120418 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0506 03:16:22.220450 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0506 03:16:22.320975 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0506 03:16:22.420388 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (99ms) I0506 03:16:22.521278 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (101ms) I0506 03:16:22.620694 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (99ms) I0506 03:16:22.720983 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (100ms) I0506 03:16:22.820447 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0506 03:16:22.920567 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (100ms) I0506 03:16:23.021274 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (101ms) I0506 03:16:23.120784 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0506 03:16:23.220952 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0506 03:16:23.321220 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (100ms) I0506 03:16:23.420510 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (99ms) I0506 03:16:23.521254 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (101ms) I0506 03:16:23.621198 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0506 03:16:23.720503 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0506 03:16:23.829003 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (108ms) I0506 03:16:23.927706 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (99ms) I0506 03:16:24.021594 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (94ms) I0506 03:16:24.121281 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (100ms) I0506 03:16:24.220627 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (99ms) I0506 03:16:24.320199 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0506 03:16:24.420476 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0506 03:16:24.521190 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (101ms) I0506 03:16:24.620978 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0506 03:16:24.720269 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (99ms) I0506 03:16:24.820961 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (101ms) I0506 03:16:24.920615 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (100ms) I0506 03:16:25.021200 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (101ms) I0506 03:16:25.120605 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (99ms) I0506 03:16:25.120642 1 tasks_processing.go:74] worker 1 stopped. I0506 03:16:25.120978 1 recorder.go:75] Recording config/workload_info with fingerprint=923fa5e9b1bb5300488cd5dde330afd504c883db44c432cd2b8187aad68e3087 I0506 03:16:25.120994 1 gather.go:177] gatherer "workloads" function "workload_info" took 9.679814305s to process 1 records I0506 03:16:25.121007 1 periodic.go:256] Periodic gather workloads completed in 9.68s I0506 03:16:25.121015 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0506 03:16:25.121019 1 periodic.go:209] Running conditional gatherer I0506 03:16:25.127539 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0506 03:16:25.336588 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0506 03:16:25.339199 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.236.35:9091: connect: connection refused I0506 03:16:25.339276 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0506 03:16:25.345379 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0506 03:16:25.345394 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345400 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345404 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345407 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345410 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345414 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345417 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345420 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345423 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 03:16:25.345426 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0506 03:16:25.345440 1 tasks_processing.go:45] number of workers: 3 I0506 03:16:25.345466 1 tasks_processing.go:69] worker 2 listening for tasks. I0506 03:16:25.345470 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0506 03:16:25.345473 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 03:16:25.345486 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0506 03:16:25.345489 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 03:16:25.345492 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0506 03:16:25.345497 1 tasks_processing.go:74] worker 1 stopped. I0506 03:16:25.345558 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0506 03:16:25.345572 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 930ns to process 1 records I0506 03:16:25.345605 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0506 03:16:25.345614 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.025µs to process 1 records I0506 03:16:25.345620 1 tasks_processing.go:74] worker 0 stopped. I0506 03:16:25.346353 1 tasks_processing.go:74] worker 2 stopped. I0506 03:16:25.346371 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 852.228µs to process 0 records I0506 03:16:25.346405 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0506 03:16:25.346432 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0506 03:16:25.372085 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=d3d6dfce5e0aa051571cf08c008f0f6635301e94bd9d9c7b8df3c3f3aeb99739 I0506 03:16:25.372234 1 diskrecorder.go:70] Writing 107 records to /var/lib/insights-operator/insights-2026-05-06-031625.tar.gz I0506 03:16:25.379060 1 diskrecorder.go:51] Wrote 107 records to disk in 6ms I0506 03:16:25.379097 1 periodic.go:278] Gathering cluster info every 2h0m0s I0506 03:16:25.379117 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0506 03:17:31.826234 1 diskrecorder.go:223] Found files to send: insights-2026-05-06-031625.tar.gz I0506 03:17:31.826266 1 insightsuploader.go:150] Checking archives to upload periodically every 16m2.661955387s I0506 03:17:31.826276 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0506 03:17:31.844961 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0506 03:17:32.099202 1 requests.go:87] Successfully reported id=2026-05-06T03:17:31Z x-rh-insights-request-id=b7770d8379d44ec09fb73331cd243341, wrote=49580 I0506 03:17:32.099262 1 insightsuploader.go:187] Uploaded report successfully in 272.976907ms I0506 03:17:32.099285 1 controller.go:128] Initializing last reported time to 2026-05-06T03:17:31Z I0506 03:17:32.099316 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0506 03:17:32.099324 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0506 03:17:32.099331 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0506 03:17:32.105833 1 controller.go:512] The operator is healthy I0506 03:18:01.841963 1 controller.go:512] The operator is healthy I0506 03:18:33.286410 1 insightsreport.go:137] Pulling report from smart-proxy I0506 03:18:33.286445 1 insightsreport.go:149] Retrieving report I0506 03:18:33.296380 1 requests.go:110] Retrieving report for cluster: 883c2c91-13a1-4c8b-b67e-00dd2a9df351 I0506 03:18:33.296394 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/883c2c91-13a1-4c8b-b67e-00dd2a9df351/reports I0506 03:18:33.299168 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/883c2c91-13a1-4c8b-b67e-00dd2a9df351/reports I0506 03:18:33.651477 1 insightsreport.go:184] Report retrieved I0506 03:18:33.677597 1 insightsreport.go:239] Report retrieved correctly I0506 03:20:01.842419 1 controller.go:512] The operator is healthy I0506 03:21:01.817239 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 03:21:01.824477 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 03:21:01.824496 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 03:21:01.829664 1 secretconfigobserver.go:119] support secret does not exist I0506 03:22:01.844298 1 controller.go:512] The operator is healthy I0506 03:24:01.842295 1 controller.go:512] The operator is healthy I0506 03:26:01.830747 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 03:26:01.838669 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 03:26:01.881726 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 03:26:01.881744 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 03:26:01.881967 1 controller.go:512] The operator is healthy I0506 03:26:01.897627 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 03:26:01.902109 1 secretconfigobserver.go:119] support secret does not exist I0506 03:28:01.842213 1 controller.go:512] The operator is healthy I0506 03:30:01.842440 1 controller.go:512] The operator is healthy I0506 03:31:01.902356 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 03:31:01.908332 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 03:31:01.908351 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 03:31:01.914956 1 secretconfigobserver.go:119] support secret does not exist I0506 03:32:01.845658 1 controller.go:512] The operator is healthy I0506 03:33:39.762765 1 insightsuploader.go:146] Nothing to report since 2026-05-06T03:17:31Z I0506 03:34:01.840986 1 controller.go:512] The operator is healthy I0506 03:36:01.817240 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 03:36:01.822738 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 03:36:01.842197 1 controller.go:512] The operator is healthy I0506 03:36:01.915491 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 03:36:01.921507 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 03:36:01.921523 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 03:36:01.926734 1 secretconfigobserver.go:119] support secret does not exist I0506 03:38:01.840059 1 controller.go:512] The operator is healthy