I0424 15:12:36.345171 1 cmd.go:253] Using service-serving-cert provided certificates I0424 15:12:36.345598 1 observer_polling.go:159] Starting file observer I0424 15:12:36.701230 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0424 15:12:36.701424 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0424 15:12:36.702003 1 secure_serving.go:57] Forcing use of http/1.1 only I0424 15:12:36.702034 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0424 15:12:36.702037 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0424 15:12:36.702072 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0424 15:12:36.702098 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0424 15:12:36.702105 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0424 15:12:36.702108 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0424 15:12:36.702112 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0424 15:12:36.705230 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0424 15:12:36.705261 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"09e60a25-2bd9-471d-9a68-da9d3a1b1be8", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0424 15:12:36.707307 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0424 15:12:36.707329 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0424 15:12:36.707342 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0424 15:12:36.707333 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0424 15:12:36.707326 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0424 15:12:36.707373 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0424 15:12:36.707783 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0424 15:12:36.708593 1 secure_serving.go:213] Serving securely on [::]:8443 I0424 15:12:36.708621 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0424 15:12:36.710418 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0424 15:12:36.710443 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0424 15:12:36.710539 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0424 15:12:36.716402 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:12:36.716419 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0424 15:12:36.720656 1 secretconfigobserver.go:119] support secret does not exist I0424 15:12:36.724845 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:12:36.728700 1 secretconfigobserver.go:119] support secret does not exist I0424 15:12:36.732076 1 recorder.go:161] Pruning old reports every 5h24m53s, max age is 288h0m0s I0424 15:12:36.736499 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0424 15:12:36.736504 1 periodic.go:209] Running clusterconfig gatherer I0424 15:12:36.736516 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0424 15:12:36.736519 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0424 15:12:36.736525 1 insightsreport.go:296] Starting report retriever I0424 15:12:36.736531 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0424 15:12:36.736556 1 tasks_processing.go:45] number of workers: 64 I0424 15:12:36.736592 1 tasks_processing.go:69] worker 4 listening for tasks. I0424 15:12:36.736599 1 tasks_processing.go:69] worker 16 listening for tasks. I0424 15:12:36.736608 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 15:12:36.736614 1 tasks_processing.go:71] worker 16 working on machine_sets task. I0424 15:12:36.736615 1 tasks_processing.go:69] worker 11 listening for tasks. I0424 15:12:36.736619 1 tasks_processing.go:69] worker 10 listening for tasks. I0424 15:12:36.736616 1 tasks_processing.go:69] worker 32 listening for tasks. I0424 15:12:36.736627 1 tasks_processing.go:71] worker 10 working on overlapping_namespace_uids task. I0424 15:12:36.736629 1 tasks_processing.go:69] worker 6 listening for tasks. I0424 15:12:36.736631 1 tasks_processing.go:71] worker 32 working on networks task. I0424 15:12:36.736638 1 tasks_processing.go:69] worker 7 listening for tasks. I0424 15:12:36.736642 1 tasks_processing.go:69] worker 8 listening for tasks. I0424 15:12:36.736641 1 tasks_processing.go:69] worker 3 listening for tasks. I0424 15:12:36.736647 1 tasks_processing.go:69] worker 9 listening for tasks. I0424 15:12:36.736608 1 tasks_processing.go:69] worker 5 listening for tasks. I0424 15:12:36.736655 1 tasks_processing.go:69] worker 25 listening for tasks. I0424 15:12:36.736655 1 tasks_processing.go:69] worker 48 listening for tasks. I0424 15:12:36.736661 1 tasks_processing.go:69] worker 41 listening for tasks. I0424 15:12:36.736661 1 tasks_processing.go:69] worker 40 listening for tasks. I0424 15:12:36.736657 1 tasks_processing.go:69] worker 2 listening for tasks. I0424 15:12:36.736668 1 tasks_processing.go:69] worker 33 listening for tasks. I0424 15:12:36.736668 1 tasks_processing.go:69] worker 13 listening for tasks. I0424 15:12:36.736677 1 tasks_processing.go:69] worker 42 listening for tasks. I0424 15:12:36.736669 1 tasks_processing.go:69] worker 17 listening for tasks. I0424 15:12:36.736680 1 tasks_processing.go:69] worker 12 listening for tasks. I0424 15:12:36.736630 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 15:12:36.736686 1 tasks_processing.go:69] worker 19 listening for tasks. I0424 15:12:36.736689 1 tasks_processing.go:69] worker 47 listening for tasks. I0424 15:12:36.736691 1 tasks_processing.go:69] worker 36 listening for tasks. I0424 15:12:36.736694 1 tasks_processing.go:69] worker 38 listening for tasks. I0424 15:12:36.736697 1 tasks_processing.go:69] worker 39 listening for tasks. I0424 15:12:36.736697 1 tasks_processing.go:69] worker 45 listening for tasks. I0424 15:12:36.736706 1 tasks_processing.go:71] worker 7 working on machine_configs task. I0424 15:12:36.736707 1 tasks_processing.go:69] worker 15 listening for tasks. I0424 15:12:36.736710 1 tasks_processing.go:71] worker 45 working on openstack_version task. I0424 15:12:36.736711 1 tasks_processing.go:69] worker 14 listening for tasks. I0424 15:12:36.736714 1 tasks_processing.go:69] worker 22 listening for tasks. I0424 15:12:36.736709 1 tasks_processing.go:69] worker 31 listening for tasks. I0424 15:12:36.736720 1 tasks_processing.go:71] worker 13 working on storage_classes task. I0424 15:12:36.736697 1 tasks_processing.go:69] worker 20 listening for tasks. I0424 15:12:36.736725 1 tasks_processing.go:71] worker 42 working on schedulers task. I0424 15:12:36.736728 1 tasks_processing.go:71] worker 41 working on infrastructures task. I0424 15:12:36.736731 1 tasks_processing.go:69] worker 54 listening for tasks. I0424 15:12:36.736732 1 tasks_processing.go:69] worker 37 listening for tasks. I0424 15:12:36.736740 1 tasks_processing.go:71] worker 9 working on image_pruners task. I0424 15:12:36.736743 1 tasks_processing.go:69] worker 55 listening for tasks. I0424 15:12:36.736744 1 tasks_processing.go:69] worker 56 listening for tasks. I0424 15:12:36.736741 1 tasks_processing.go:69] worker 53 listening for tasks. I0424 15:12:36.736752 1 tasks_processing.go:69] worker 61 listening for tasks. I0424 15:12:36.736732 1 tasks_processing.go:71] worker 3 working on ceph_cluster task. I0424 15:12:36.736757 1 tasks_processing.go:71] worker 40 working on machine_config_pools task. I0424 15:12:36.736760 1 tasks_processing.go:69] worker 58 listening for tasks. I0424 15:12:36.736757 1 tasks_processing.go:69] worker 63 listening for tasks. I0424 15:12:36.736761 1 tasks_processing.go:69] worker 51 listening for tasks. I0424 15:12:36.736767 1 tasks_processing.go:69] worker 59 listening for tasks. I0424 15:12:36.736774 1 tasks_processing.go:69] worker 60 listening for tasks. I0424 15:12:36.736615 1 tasks_processing.go:71] worker 0 working on machine_autoscalers task. I0424 15:12:36.736777 1 tasks_processing.go:69] worker 50 listening for tasks. I0424 15:12:36.736782 1 tasks_processing.go:71] worker 19 working on openstack_dataplanedeployments task. I0424 15:12:36.736781 1 tasks_processing.go:71] worker 12 working on openshift_logging task. I0424 15:12:36.736690 1 tasks_processing.go:69] worker 27 listening for tasks. I0424 15:12:36.736791 1 tasks_processing.go:71] worker 50 working on dvo_metrics task. I0424 15:12:36.736793 1 tasks_processing.go:71] worker 27 working on aggregated_monitoring_cr_names task. I0424 15:12:36.736834 1 tasks_processing.go:71] worker 47 working on sap_pods task. I0424 15:12:36.736683 1 tasks_processing.go:69] worker 26 listening for tasks. I0424 15:12:36.736684 1 tasks_processing.go:69] worker 46 listening for tasks. I0424 15:12:36.736683 1 tasks_processing.go:69] worker 35 listening for tasks. I0424 15:12:36.736674 1 tasks_processing.go:69] worker 29 listening for tasks. I0424 15:12:36.736688 1 tasks_processing.go:69] worker 30 listening for tasks. I0424 15:12:36.736676 1 tasks_processing.go:69] worker 34 listening for tasks. I0424 15:12:36.736677 1 tasks_processing.go:69] worker 18 listening for tasks. I0424 15:12:36.736700 1 tasks_processing.go:71] worker 38 working on validating_webhook_configurations task. I0424 15:12:36.736622 1 tasks_processing.go:71] worker 11 working on feature_gates task. I0424 15:12:36.737020 1 tasks_processing.go:71] worker 18 working on ingress_certificates task. I0424 15:12:36.736704 1 tasks_processing.go:71] worker 39 working on config_maps task. I0424 15:12:36.736702 1 tasks_processing.go:71] worker 6 working on monitoring_persistent_volumes task. I0424 15:12:36.736698 1 tasks_processing.go:71] worker 36 working on cost_management_metrics_configs task. I0424 15:12:36.737485 1 tasks_processing.go:71] worker 15 working on nodes task. I0424 15:12:36.737566 1 tasks_processing.go:71] worker 56 working on tsdb_status task. I0424 15:12:36.737645 1 tasks_processing.go:71] worker 30 working on qemu_kubevirt_launcher_logs task. I0424 15:12:36.737682 1 tasks_processing.go:71] worker 63 working on machines task. I0424 15:12:36.737671 1 tasks_processing.go:71] worker 46 working on service_accounts task. I0424 15:12:36.737520 1 tasks_processing.go:71] worker 29 working on operators_pods_and_events task. I0424 15:12:36.737872 1 tasks_processing.go:71] worker 54 working on sap_config task. I0424 15:12:36.737925 1 tasks_processing.go:71] worker 61 working on openstack_controlplanes task. I0424 15:12:36.736699 1 tasks_processing.go:69] worker 43 listening for tasks. I0424 15:12:36.738135 1 tasks_processing.go:71] worker 43 working on node_logs task. I0424 15:12:36.738176 1 tasks_processing.go:71] worker 58 working on container_images task. I0424 15:12:36.736713 1 tasks_processing.go:71] worker 17 working on nodenetworkstates task. I0424 15:12:36.737956 1 tasks_processing.go:71] worker 53 working on clusterroles task. I0424 15:12:36.737708 1 tasks_processing.go:71] worker 34 working on nodenetworkconfigurationpolicies task. I0424 15:12:36.737577 1 tasks_processing.go:71] worker 35 working on container_runtime_configs task. I0424 15:12:36.736712 1 tasks_processing.go:71] worker 2 working on active_alerts task. I0424 15:12:36.736716 1 tasks_processing.go:71] worker 33 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0424 15:12:36.736717 1 tasks_processing.go:69] worker 57 listening for tasks. I0424 15:12:36.736716 1 tasks_processing.go:71] worker 5 working on silenced_alerts task. I0424 15:12:36.736718 1 tasks_processing.go:71] worker 25 working on sap_datahubs task. I0424 15:12:36.736724 1 tasks_processing.go:71] worker 48 working on install_plans task. I0424 15:12:36.736725 1 tasks_processing.go:69] worker 23 listening for tasks. I0424 15:12:36.736724 1 tasks_processing.go:69] worker 49 listening for tasks. I0424 15:12:36.736732 1 tasks_processing.go:69] worker 24 listening for tasks. I0424 15:12:36.736733 1 tasks_processing.go:71] worker 8 working on ingress task. I0424 15:12:36.736747 1 tasks_processing.go:69] worker 52 listening for tasks. I0424 15:12:36.736754 1 tasks_processing.go:71] worker 4 working on mutating_webhook_configurations task. I0424 15:12:36.736765 1 tasks_processing.go:69] worker 62 listening for tasks. I0424 15:12:36.736777 1 tasks_processing.go:71] worker 1 working on pod_network_connectivity_checks task. I0424 15:12:36.737587 1 tasks_processing.go:71] worker 26 working on openstack_dataplanenodesets task. I0424 15:12:36.737720 1 tasks_processing.go:71] worker 59 working on crds task. I0424 15:12:36.737751 1 tasks_processing.go:71] worker 51 working on image_registries task. I0424 15:12:36.736706 1 tasks_processing.go:69] worker 44 listening for tasks. I0424 15:12:36.739536 1 tasks_processing.go:71] worker 44 working on cluster_apiserver task. I0424 15:12:36.737762 1 tasks_processing.go:71] worker 60 working on machine_healthchecks task. I0424 15:12:36.739565 1 tasks_processing.go:71] worker 24 working on operators task. I0424 15:12:36.739600 1 tasks_processing.go:71] worker 49 working on storage_cluster task. I0424 15:12:36.737771 1 tasks_processing.go:71] worker 37 working on authentication task. I0424 15:12:36.739742 1 tasks_processing.go:71] worker 16 working on lokistack task. I0424 15:12:36.739755 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 3.117697ms to process 0 records I0424 15:12:36.739797 1 tasks_processing.go:71] worker 57 working on proxies task. I0424 15:12:36.737801 1 tasks_processing.go:71] worker 20 working on support_secret task. I0424 15:12:36.739944 1 tasks_processing.go:71] worker 23 working on metrics task. I0424 15:12:36.737851 1 tasks_processing.go:71] worker 55 working on version task. I0424 15:12:36.740037 1 tasks_processing.go:71] worker 52 working on certificate_signing_requests task. I0424 15:12:36.737866 1 tasks_processing.go:71] worker 14 working on olm_operators task. I0424 15:12:36.736706 1 tasks_processing.go:69] worker 28 listening for tasks. I0424 15:12:36.740218 1 tasks_processing.go:71] worker 28 working on pdbs task. I0424 15:12:36.738023 1 tasks_processing.go:71] worker 22 working on openshift_machine_api_events task. I0424 15:12:36.736705 1 tasks_processing.go:69] worker 21 listening for tasks. I0424 15:12:36.738033 1 tasks_processing.go:71] worker 31 working on image task. I0424 15:12:36.740169 1 tasks_processing.go:71] worker 62 working on jaegers task. I0424 15:12:36.740292 1 tasks_processing.go:71] worker 21 working on oauths task. I0424 15:12:36.741669 1 tasks_processing.go:74] worker 19 stopped. I0424 15:12:36.741685 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 4.874887ms to process 0 records E0424 15:12:36.744078 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:57564->172.30.0.10:53: read: connection refused I0424 15:12:36.744097 1 tasks_processing.go:74] worker 56 stopped. E0424 15:12:36.744090 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:57564->172.30.0.10:53: read: connection refused E0424 15:12:36.744104 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:57564->172.30.0.10:53: read: connection refused I0424 15:12:36.744114 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 6.474235ms to process 0 records E0424 15:12:36.744111 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:47205->172.30.0.10:53: read: connection refused E0424 15:12:36.744116 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:47205->172.30.0.10:53: read: connection refused E0424 15:12:36.744122 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:57564->172.30.0.10:53: read: connection refused I0424 15:12:36.744134 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 4.154576ms to process 0 records E0424 15:12:36.744144 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:47205->172.30.0.10:53: read: connection refused I0424 15:12:36.744154 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 5.052826ms to process 0 records E0424 15:12:36.744162 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:47205->172.30.0.10:53: read: connection refused I0424 15:12:36.744170 1 tasks_processing.go:74] worker 5 stopped. I0424 15:12:36.744171 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 5.070231ms to process 0 records I0424 15:12:36.744176 1 tasks_processing.go:74] worker 2 stopped. I0424 15:12:36.744164 1 tasks_processing.go:74] worker 23 stopped. I0424 15:12:36.747082 1 tasks_processing.go:74] worker 3 stopped. I0424 15:12:36.747100 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 10.320932ms to process 0 records I0424 15:12:36.747144 1 tasks_processing.go:74] worker 0 stopped. I0424 15:12:36.747156 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 10.359713ms to process 0 records I0424 15:12:36.747210 1 tasks_processing.go:74] worker 45 stopped. I0424 15:12:36.747223 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 10.492902ms to process 0 records I0424 15:12:36.747245 1 tasks_processing.go:74] worker 17 stopped. I0424 15:12:36.747257 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 8.911516ms to process 0 records I0424 15:12:36.747280 1 tasks_processing.go:74] worker 12 stopped. I0424 15:12:36.747290 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 10.487747ms to process 0 records I0424 15:12:36.747299 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 10.432289ms to process 0 records I0424 15:12:36.747305 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 9.814031ms to process 0 records I0424 15:12:36.747311 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 9.298044ms to process 0 records I0424 15:12:36.747317 1 tasks_processing.go:74] worker 61 stopped. I0424 15:12:36.747323 1 tasks_processing.go:74] worker 47 stopped. I0424 15:12:36.747328 1 tasks_processing.go:74] worker 36 stopped. I0424 15:12:36.747346 1 tasks_processing.go:74] worker 42 stopped. I0424 15:12:36.747424 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=5a1c912616735d868e02643a39572b00df32308512ed4fe5b19cb4ccaf09019b I0424 15:12:36.747436 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 10.612255ms to process 1 records I0424 15:12:36.747499 1 tasks_processing.go:74] worker 41 stopped. I0424 15:12:36.747565 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0424 15:12:36.747581 1 controller.go:326] The initial operator extension status is healthy I0424 15:12:36.747589 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0424 15:12:36.747593 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0424 15:12:36.747596 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0424 15:12:36.747608 1 controller.go:489] The operator is still being initialized I0424 15:12:36.747616 1 controller.go:512] The operator is healthy I0424 15:12:36.747672 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0424 15:12:36.747678 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0424 15:12:36.747809 1 operator.go:288] started I0424 15:12:36.747820 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0424 15:12:36.748065 1 recorder.go:75] Recording config/infrastructure with fingerprint=4878ba1fe4c1b4173742d767147e286c3066edf1ea5d4e318bedfef455c591c1 I0424 15:12:36.748083 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 10.754845ms to process 1 records E0424 15:12:36.748100 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0424 15:12:36.748109 1 gather.go:177] gatherer "clusterconfig" function "machines" took 9.787414ms to process 0 records I0424 15:12:36.748168 1 tasks_processing.go:74] worker 63 stopped. I0424 15:12:36.748235 1 tasks_processing.go:74] worker 32 stopped. I0424 15:12:36.748260 1 recorder.go:75] Recording config/network with fingerprint=c6c29cc4fd5437fdac435aa0b4a65090dfe5ddea4b1a45b8613dbaad0c5cf16c I0424 15:12:36.748274 1 gather.go:177] gatherer "clusterconfig" function "networks" took 10.887279ms to process 1 records I0424 15:12:36.748354 1 tasks_processing.go:74] worker 11 stopped. I0424 15:12:36.748358 1 recorder.go:75] Recording config/featuregate with fingerprint=6dbc3c75f6f67e57a4b42f3bad655d48deb6438e619eddf5e8b8128a93b552e6 I0424 15:12:36.748369 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 10.650429ms to process 1 records I0424 15:12:36.748450 1 tasks_processing.go:74] worker 9 stopped. I0424 15:12:36.748563 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=db85cd371c2c26ddecd63c27bfd392501bbdba425c4eed6dbeacd5566e125acc I0424 15:12:36.748574 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 10.95473ms to process 1 records I0424 15:12:36.750804 1 tasks_processing.go:74] worker 6 stopped. I0424 15:12:36.750817 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 13.344286ms to process 0 records I0424 15:12:36.751680 1 tasks_processing.go:74] worker 40 stopped. I0424 15:12:36.751740 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 14.912842ms to process 0 records I0424 15:12:36.753402 1 tasks_processing.go:74] worker 10 stopped. I0424 15:12:36.753426 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0424 15:12:36.753434 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 16.760083ms to process 1 records I0424 15:12:36.756679 1 gather_logs.go:145] no pods in namespace were found I0424 15:12:36.756692 1 tasks_processing.go:74] worker 30 stopped. I0424 15:12:36.756699 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 19.006918ms to process 0 records I0424 15:12:36.756726 1 tasks_processing.go:74] worker 43 stopped. I0424 15:12:36.756741 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 18.556542ms to process 0 records I0424 15:12:36.756922 1 tasks_processing.go:74] worker 25 stopped. I0424 15:12:36.756942 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 17.84359ms to process 0 records I0424 15:12:36.756958 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 17.802988ms to process 0 records I0424 15:12:36.756966 1 tasks_processing.go:74] worker 26 stopped. I0424 15:12:36.757007 1 tasks_processing.go:74] worker 13 stopped. I0424 15:12:36.757062 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=9d0c92fefb47d3822f5c5fdb02f19765cecc3257620b80d9adc7301bd3cfada5 I0424 15:12:36.757079 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=452e0a330c602656125fc17ba046ce59c3fdfaf77eae146c7b92eee35b6fedf3 I0424 15:12:36.757085 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 20.236587ms to process 2 records I0424 15:12:36.757095 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 19.095072ms to process 0 records I0424 15:12:36.757102 1 tasks_processing.go:74] worker 54 stopped. I0424 15:12:36.762301 1 tasks_processing.go:74] worker 62 stopped. I0424 15:12:36.762312 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 21.969779ms to process 0 records I0424 15:12:36.762378 1 tasks_processing.go:74] worker 34 stopped. I0424 15:12:36.762390 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 23.435379ms to process 0 records I0424 15:12:36.762403 1 tasks_processing.go:74] worker 15 stopped. I0424 15:12:36.762692 1 recorder.go:75] Recording config/node/ip-10-0-0-207.ec2.internal with fingerprint=03c21e41b39f0888a843737e6d3ba4404625e57862b2c222fdcdd509198292f8 I0424 15:12:36.762748 1 recorder.go:75] Recording config/node/ip-10-0-1-63.ec2.internal with fingerprint=d98c4ca59688fbc24153bdc68ec3ce9438b3ed66bf943ccfea7d77f52be80aad I0424 15:12:36.762756 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 24.866482ms to process 2 records E0424 15:12:36.762763 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0424 15:12:36.762768 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 22.880377ms to process 0 records I0424 15:12:36.762773 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 22.904284ms to process 0 records I0424 15:12:36.762778 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 22.774434ms to process 0 records I0424 15:12:36.762802 1 tasks_processing.go:74] worker 49 stopped. I0424 15:12:36.762811 1 tasks_processing.go:74] worker 60 stopped. I0424 15:12:36.762813 1 tasks_processing.go:74] worker 16 stopped. I0424 15:12:36.762879 1 tasks_processing.go:74] worker 44 stopped. I0424 15:12:36.762897 1 recorder.go:75] Recording config/apiserver with fingerprint=ed059f41b3bc561d3cc2896b6353292c5bd619a03f70698aadc3052dcfafcc27 I0424 15:12:36.762904 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 23.007623ms to process 1 records I0424 15:12:36.762983 1 tasks_processing.go:74] worker 8 stopped. I0424 15:12:36.763037 1 recorder.go:75] Recording config/ingress with fingerprint=7376c99ac965a557413dbc393318de25abf0a8c379e9bea93ccf9f13df44d553 I0424 15:12:36.763047 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 23.563674ms to process 1 records I0424 15:12:36.763055 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 23.784999ms to process 0 records I0424 15:12:36.763061 1 tasks_processing.go:74] worker 35 stopped. I0424 15:12:36.769808 1 tasks_processing.go:74] worker 1 stopped. E0424 15:12:36.769822 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0424 15:12:36.769832 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 30.677261ms to process 0 records I0424 15:12:36.770521 1 tasks_processing.go:74] worker 37 stopped. I0424 15:12:36.771349 1 recorder.go:75] Recording config/authentication with fingerprint=4c0393e3fb5868400a2004e575fdc3166dec868a305a0ca95cc704e0ffb21fd6 I0424 15:12:36.771418 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 30.798174ms to process 1 records I0424 15:12:36.771771 1 tasks_processing.go:74] worker 21 stopped. I0424 15:12:36.771961 1 recorder.go:75] Recording config/oauth with fingerprint=b29453ec67a9ba1e4db42d38646a574699c30653b89a5f4709e6e5208b38dc93 I0424 15:12:36.772022 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 30.221359ms to process 1 records I0424 15:12:36.772168 1 recorder.go:75] Recording config/image with fingerprint=f08504cb45c669d94c5ed1a10288bf5d1966f1ec54c20ad55c0c9ea5ad661a9e I0424 15:12:36.772225 1 gather.go:177] gatherer "clusterconfig" function "image" took 30.640189ms to process 1 records E0424 15:12:36.772240 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0424 15:12:36.772255 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 31.264033ms to process 0 records I0424 15:12:36.772180 1 tasks_processing.go:74] worker 31 stopped. I0424 15:12:36.772267 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 31.359279ms to process 0 records I0424 15:12:36.772273 1 tasks_processing.go:74] worker 20 stopped. I0424 15:12:36.772277 1 tasks_processing.go:74] worker 22 stopped. I0424 15:12:36.772474 1 recorder.go:75] Recording config/proxy with fingerprint=a2d9043356a73e6cd75f20815b43705345726e901d80672b25766d5125ed45e6 I0424 15:12:36.772509 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 32.509692ms to process 1 records I0424 15:12:36.772447 1 tasks_processing.go:74] worker 57 stopped. I0424 15:12:36.774174 1 tasks_processing.go:74] worker 27 stopped. I0424 15:12:36.774191 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 37.371596ms to process 0 records I0424 15:12:36.775063 1 tasks_processing.go:74] worker 28 stopped. I0424 15:12:36.775129 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0424 15:12:36.775146 1 controller.go:212] Source scaController *sca.Controller is not ready I0424 15:12:36.775151 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0424 15:12:36.775156 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0424 15:12:36.775162 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0424 15:12:36.775184 1 controller.go:489] The operator is still being initialized I0424 15:12:36.775196 1 controller.go:512] The operator is healthy I0424 15:12:36.775247 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=350def5ba17df8754d9655d243006d9b651a11e144513f2794a6a73b9b4381c7 I0424 15:12:36.775320 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=bc49e350f3d726c3c7bd4f1fd841f001aac9a94ac093010273da4aa345b372fb I0424 15:12:36.775345 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=169b3811135f8fd6fad0e2da95c025b4df9e9cd5f352a02d43220f9b50561fbc I0424 15:12:36.775370 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=74513bde2a575cdc6f8653aa666ef35104120447598b2d73d8dc472f24c20d2a I0424 15:12:36.775419 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=964e89048ab5008b0df28b176ec7a003df493388c98a78ff115684d3d62d5ff8 I0424 15:12:36.775450 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=0a9e45625c6c9051c5d7fb9791e0fabf52c44486759d3b13abc7492a3d9ed502 I0424 15:12:36.775462 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 34.830281ms to process 6 records I0424 15:12:36.775566 1 tasks_processing.go:74] worker 4 stopped. I0424 15:12:36.775695 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=89e593bb470dabf97ea22cce8ec3353c53c3f5047dafeed0190d034c172b816b I0424 15:12:36.775732 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=72be4aaed54becd69205e637459c7ac3e7542e7e0148d8a4cd21e540359e22a7 I0424 15:12:36.775755 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=1d3e33b165e6c432b9bab5e9a5472484bc1fb28e80fd3ee0767d1de1d1495040 I0424 15:12:36.775763 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 35.979827ms to process 3 records W0424 15:12:36.775845 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:12:36.775887 1 tasks_processing.go:74] worker 51 stopped. I0424 15:12:36.776090 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=f8f2f3e9e5960426a7132a8ef07af88f1d1400fe90ef5f6920aac19435fc9c48 I0424 15:12:36.776100 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 36.049816ms to process 1 records I0424 15:12:36.776175 1 tasks_processing.go:74] worker 38 stopped. I0424 15:12:36.776181 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=9a537acb59d47e496a2675872e9d00e8dfc1abfae54faadc1a46c41de79257fb I0424 15:12:36.776254 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=28e9ab428079016c255f7cd884f043656c7d4372358353359782bd8297958f42 I0424 15:12:36.776280 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=8b4ed49a7e5b3278ee8d3901cd501c88225bf23b3da23ff1a43fde67ae46bcde I0424 15:12:36.776308 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=045e16da0fc2b764ca1bae940bb1b09649208c1f6498ae0a88b611fc2c910fef I0424 15:12:36.776331 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=49ea49a6d651fe185024ff46594992a2f96e2aedcf5ac7cc345a2593b0ef53b5 I0424 15:12:36.776352 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=76f81b6de7170b996541ebafce0a24a7d42ed60ed56d7cb73c886497d41d41c4 I0424 15:12:36.776375 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=401c674dc3c0b801849a0faec69352c70716e4f541571a2bcc1a02b24e88ce05 I0424 15:12:36.776408 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=bca46b023737404dbf307c160d861ee4a96094ef6d50e85c931db7f9e463ddcd I0424 15:12:36.776429 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=a8a27ea4f3d2d0d850b2f3ee58cda4471112cfaa081a3591251bb3b5fb9e8d30 I0424 15:12:36.776465 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=c06a18043b1e7ce0360410077ec7dea38675fcbc8cc8d7883918a7ae34954cf4 I0424 15:12:36.776490 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=2737dc24b35402ecffb93cfc73464fa392099e86f5f1092242642dd9c6c78e58 I0424 15:12:36.776498 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 39.046903ms to process 11 records I0424 15:12:36.779914 1 tasks_processing.go:74] worker 52 stopped. I0424 15:12:36.779935 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 39.856045ms to process 0 records I0424 15:12:36.780387 1 tasks_processing.go:74] worker 53 stopped. I0424 15:12:36.780539 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=abe1a31d536ec45d7364c780e4c45a43863bfda6b1ffa01b8ce5216e6ffaf249 I0424 15:12:36.780629 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=28662984d32210caa91624590537eccc4e712da2df098100352fc754fd9c8048 I0424 15:12:36.780637 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 41.656171ms to process 2 records I0424 15:12:36.781048 1 tasks_processing.go:74] worker 7 stopped. I0424 15:12:36.781078 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0424 15:12:36.781094 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 44.331488ms to process 1 records I0424 15:12:36.783272 1 tasks_processing.go:74] worker 55 stopped. I0424 15:12:36.784231 1 recorder.go:75] Recording config/version with fingerprint=2244063423600dbb5535d0910de92d9ec3da0dd7b87b8421ba253415c00404f2 I0424 15:12:36.784297 1 recorder.go:75] Recording config/id with fingerprint=22b1a1a1fd88287fed35eb0ee99fd3c3410180388be5cc92b3af47606b85307a I0424 15:12:36.784328 1 gather.go:177] gatherer "clusterconfig" function "version" took 43.291485ms to process 2 records I0424 15:12:36.784614 1 tasks_processing.go:74] worker 59 stopped. I0424 15:12:36.785894 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=1f25bc9f6ccba3becd69afc803470a1654c9ae16fc831f16f48e6de0177f0f5d I0424 15:12:36.786240 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=3242f24c9a93daacad6e9fdc75e31ea4cb3f6f2f34c2875320def4b3bf02bd58 I0424 15:12:36.786254 1 gather.go:177] gatherer "clusterconfig" function "crds" took 44.058837ms to process 2 records I0424 15:12:36.786462 1 tasks_processing.go:74] worker 58 stopped. E0424 15:12:36.786837 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27732cbcc8-2927-455e-b585-6703246bc32e%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:46883->172.30.0.10:53: read: connection refused I0424 15:12:36.786857 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27732cbcc8-2927-455e-b585-6703246bc32e%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:46883->172.30.0.10:53: read: connection refused I0424 15:12:36.787690 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl with fingerprint=0e588c7c20088cd5bc8db084d1eccfc9ae59204655347f6a3266b36d12c50765 I0424 15:12:36.787802 1 recorder.go:75] Recording config/pod/openshift-multus/multus-q89wb with fingerprint=64ce5cfa977558dea759e8e0fa5c467ad713c65ba5fdba0600deb78f7fbf0acd I0424 15:12:36.788052 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-4r57z with fingerprint=3330dbcb38f0172e85d76cc35f8592b0e7e5008d021e1e02a7328886bec84b39 I0424 15:12:36.788121 1 recorder.go:75] Recording config/running_containers with fingerprint=a843d4b8a4de86a22a427a214fadacdcb7cef58b73f3f60a7cfa03b900e11e0e I0424 15:12:36.788131 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 48.26877ms to process 4 records I0424 15:12:36.788175 1 recorder.go:75] Recording config/olm_operators with fingerprint=9db2cfa10831a86111f58964b5392920cdb08cf3a766dd6f8daf66e5d6737d1e I0424 15:12:36.788184 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 47.013822ms to process 1 records I0424 15:12:36.788188 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 48.816009ms to process 0 records I0424 15:12:36.788196 1 tasks_processing.go:74] worker 33 stopped. I0424 15:12:36.788200 1 tasks_processing.go:74] worker 14 stopped. I0424 15:12:36.789099 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0424 15:12:36.792253 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:55405->172.30.0.10:53: read: connection refused I0424 15:12:36.792265 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:55405->172.30.0.10:53: read: connection refused I0424 15:12:36.795776 1 tasks_processing.go:74] worker 39 stopped. E0424 15:12:36.795791 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0424 15:12:36.795798 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0424 15:12:36.795804 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0424 15:12:36.795815 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0424 15:12:36.795857 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0424 15:12:36.795867 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0424 15:12:36.795873 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0424 15:12:36.795888 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0424 15:12:36.795934 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0424 15:12:36.795945 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0424 15:12:36.795954 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 58.316291ms to process 7 records I0424 15:12:36.797806 1 tasks_processing.go:74] worker 18 stopped. E0424 15:12:36.797825 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0424 15:12:36.797833 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2psja02f2k2pd7f83gt8atnjlu4e5521-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2psja02f2k2pd7f83gt8atnjlu4e5521-primary-cert-bundle-secret" not found I0424 15:12:36.797886 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=9c9826d4d06d6154c3aa0a42eda52dbc8caa55c1c402dadc30a0b9942b6e8ebf I0424 15:12:36.797923 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 60.763117ms to process 1 records I0424 15:12:36.807553 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0424 15:12:36.807562 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0424 15:12:36.807553 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0424 15:12:36.810691 1 base_controller.go:82] Caches are synced for ConfigController I0424 15:12:36.810704 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0424 15:12:36.816470 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:12:36.848221 1 base_controller.go:82] Caches are synced for LoggingSyncer I0424 15:12:36.848254 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0424 15:12:37.774032 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:12:38.383044 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 21 containers I0424 15:12:38.383056 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0424 15:12:38.383114 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-4dk46 pod in namespace openshift-network-console (previous: false). I0424 15:12:38.590380 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-4dppw pod in namespace openshift-network-console (previous: false). I0424 15:12:38.598923 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0424 15:12:38.773477 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:12:38.789579 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-gvlhl pod in namespace openshift-console-operator (previous: true). I0424 15:12:38.989451 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-gvlhl pod in namespace openshift-console-operator (previous: false). I0424 15:12:39.195743 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-647ccd467c-w4cc9 pod in namespace openshift-console (previous: false). I0424 15:12:39.206124 1 tasks_processing.go:74] worker 24 stopped. I0424 15:12:39.206248 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=ec55e03da0ff4716a0ad741e7dd950ac1a4e0fb8d75ec73f490b64ecbce41bdf I0424 15:12:39.206305 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=b1d3a57201f472fba248e6e22732d03f4ebac77f3d7204455295e9c6f31f376a I0424 15:12:39.206340 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=90516ed105d8e72e95a6c3403c7b3a767d22b0101e1e4fd59a9b1557c205efa2 I0424 15:12:39.206357 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0424 15:12:39.206389 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=d6a7a521a3c9e38e6f2a552844f66f5d4d2a8fe57f13bc6b2051aae344799a3a I0424 15:12:39.206415 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0424 15:12:39.206448 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=802d3784185551266005eafb24582122a271127871a18141bee7223c83bcb4e2 I0424 15:12:39.206487 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=20f020aded283d2d4707b80c3f171c381a2ad9e4097b0769275d407d47b89c09 I0424 15:12:39.206521 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=9830e3c7e2c937ff4be3b2c0b3b517728086f82972165f43d1bffff9bd6e703f I0424 15:12:39.206562 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=cd51c2dcb30df11e47d998c736909880cf50cd27ea8b529bc871517fbb7d4bcd I0424 15:12:39.206825 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=63e2fc795670b77f88f07734656cb981d64cd8e0853bbd43811ab8a1395b1db3 I0424 15:12:39.206855 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=8092b138834b3c63decc1501aaf8aea35434fa5618bfc27e40d4aa5ef7408c22 I0424 15:12:39.206869 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0424 15:12:39.206894 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=d53fef0da77be11c3a52c4062885a3ae63a8bbc60453f2ea29d06d040e087b7c I0424 15:12:39.206909 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0424 15:12:39.206933 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=bf0f1f5f6deba72dc0bf0fc802e09f21c52b0f23a2a2089c1b6b28825b096814 I0424 15:12:39.206946 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0424 15:12:39.206975 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=a3c55cc484d00d48e6e92da1eb8464b7dbb9ea21452d2eda08eb602e4e517b59 I0424 15:12:39.207007 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0424 15:12:39.207049 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=ff621f4cb7112bddb02468f61e7822962174bcb67c5bc93363ea100463202157 I0424 15:12:39.207171 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=d7bac72ca8122b9fe2f1fa89c2e1993793fdf085b592b62c1a354d2b9cb49315 I0424 15:12:39.207186 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0424 15:12:39.207199 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0424 15:12:39.207227 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0424 15:12:39.207257 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=575c88d29076f27a3321c2a37bbc4963bee9ca9495edd6a1f142b269c8df91d6 I0424 15:12:39.207296 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=51bec8db13602548b5bbd6be073a52460eb362f5ecd549c34ecd54cc73d8f92c I0424 15:12:39.207317 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0424 15:12:39.207343 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=ab4e3f266fa6a4c6c5eed9fd2f7c2c3da189946f26fe120eedd50d99af693b93 I0424 15:12:39.207357 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0424 15:12:39.207378 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=a7dad644a42faf5ea4e7e1b7ae34da03bcf7b852a3067baa25da262d644c47dd I0424 15:12:39.207402 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=605898c2a06ccc3c631bd7d78efecdd2a75aa5a63db2b887e9d4c453d7e15df1 I0424 15:12:39.207449 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=efae78f2e52fed6dd1b8906bd81c2842b09df8f4d4ff204dc6be00e31c636a2b I0424 15:12:39.207477 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=7f538059156e189b4bd30fcf1e5f9a47f2d6eadbd8fc5bde45441fe9e13f004c I0424 15:12:39.207509 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=258fdd1d813c4a1153a4c493075d83835c870c804b1a4474b5a42e07191c935a I0424 15:12:39.207523 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0424 15:12:39.207560 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=9f1f8a0f8918fd813ea71ef56c129e5c71fc7b7d780744365c37e327b1b778b3 I0424 15:12:39.207585 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0424 15:12:39.207600 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0424 15:12:39.207612 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.466534461s to process 38 records I0424 15:12:39.383772 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:39.383791 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-647ccd467c-wlm9z pod in namespace openshift-console (previous: false). I0424 15:12:39.582676 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:39.582694 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-7b6d7cfd7f-5vzrt pod in namespace openshift-console (previous: false). W0424 15:12:39.774676 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:12:39.789410 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-4rv27 pod in namespace openshift-console (previous: false). I0424 15:12:39.983237 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:39.983256 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-mxmsx pod in namespace openshift-console (previous: false). I0424 15:12:40.190689 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-j4qj9 pod in namespace openshift-dns (previous: false). I0424 15:12:40.389257 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-j4qj9 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-j4qj9\" is waiting to start: ContainerCreating" I0424 15:12:40.389274 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-j4qj9\" is waiting to start: ContainerCreating" I0424 15:12:40.389282 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-j4qj9 pod in namespace openshift-dns (previous: false). I0424 15:12:40.590007 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-j4qj9 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-j4qj9\" is waiting to start: ContainerCreating" I0424 15:12:40.590024 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-j4qj9\" is waiting to start: ContainerCreating" I0424 15:12:40.590034 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-9267j pod in namespace openshift-dns (previous: false). W0424 15:12:40.774242 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 15:12:40.790013 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:40.790026 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-qmh58 pod in namespace openshift-dns (previous: false). I0424 15:12:41.006230 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:41.006243 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5868f658b9-pxv26 pod in namespace openshift-image-registry (previous: false). I0424 15:12:41.183895 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:41.183913 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5868f658b9-wt9qd pod in namespace openshift-image-registry (previous: false). I0424 15:12:41.390243 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5b75c965f4-bhmft pod in namespace openshift-image-registry (previous: false). I0424 15:12:41.582712 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:41.582729 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-k8jjt pod in namespace openshift-image-registry (previous: false). W0424 15:12:41.774305 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0424 15:12:41.774345 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0424 15:12:41.774360 1 tasks_processing.go:74] worker 50 stopped. E0424 15:12:41.774369 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0424 15:12:41.774380 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0424 15:12:41.774389 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.037556912s to process 1 records I0424 15:12:41.789362 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-s6xmg pod in namespace openshift-image-registry (previous: false). I0424 15:12:41.990060 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-55b7955667-grf4t pod in namespace openshift-ingress (previous: false). I0424 15:12:42.190933 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-774967f6d5-2xt8q pod in namespace openshift-ingress (previous: false). I0424 15:12:42.389872 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-774967f6d5-bvbxt pod in namespace openshift-ingress (previous: false). I0424 15:12:42.583433 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 15:12:42.583475 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-bthf7 pod in namespace openshift-ingress-canary (previous: false). I0424 15:12:42.789307 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-bthf7 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-bthf7\" is waiting to start: ContainerCreating" I0424 15:12:42.789324 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-bthf7\" is waiting to start: ContainerCreating" I0424 15:12:42.789341 1 tasks_processing.go:74] worker 29 stopped. I0424 15:12:42.789435 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=dcc0a6c8c8f2cd7fa42977534c00cabb82465838b5408465f90a080fb45c7b1b I0424 15:12:42.789552 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=0e7d9461d5bfbcb1d5d0a0b30e956292ed5f7712753b9c976cde7f978c18ba2c I0424 15:12:42.789603 1 recorder.go:75] Recording events/openshift-console with fingerprint=6a336356d6e1c2446c0005f7402914f24fcecda75738249e3bea3a0f154991ae I0424 15:12:42.789627 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=705ed48ef895742a031339f540df42776858f2707e4753bd84cdfd6fa028acdf I0424 15:12:42.789652 1 recorder.go:75] Recording events/openshift-dns with fingerprint=99efa3bfa2cd97037ed4bd95ce54f2e2bf1081b055583281207fced23e75a4f3 I0424 15:12:42.789758 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=bbf9a79073c371863c5f7bbc933bed745bba98cb55817159ab0f4fea9c6fc421 I0424 15:12:42.789784 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=508b6552647080918775f973f628cd2fd1d7a2280c8967424a64978f28042670 I0424 15:12:42.789846 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=14a3a5bc9318545955324273c9bd6354410bba70c4f8c6112ccb96568498736a I0424 15:12:42.789857 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=c9211cb12077f3801d2b5b2697283b6d79f23fa450f8a46eed804ad0c48cde02 I0424 15:12:42.789864 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-4dk46/networking-console-plugin_current.log with fingerprint=38099fe0efb053fb4c045acd87e9c61c8242aa90d06894d7b037508dd9446d27 I0424 15:12:42.789870 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-4dppw/networking-console-plugin_current.log with fingerprint=a8ee4f0fa9cc752b15ecfb3e1e009b2e2da9369e573d8ae8cb0cd329ce6a68e1 I0424 15:12:42.790022 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl with fingerprint=0e588c7c20088cd5bc8db084d1eccfc9ae59204655347f6a3266b36d12c50765 E0424 15:12:42.790038 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json" was already recorded and had the fingerprint "0e588c7c20088cd5bc8db084d1eccfc9ae59204655347f6a3266b36d12c50765", overwriting with the record having fingerprint "0e588c7c20088cd5bc8db084d1eccfc9ae59204655347f6a3266b36d12c50765" W0424 15:12:42.790049 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json" because of the warning: warning: the record with the same fingerprint "0e588c7c20088cd5bc8db084d1eccfc9ae59204655347f6a3266b36d12c50765" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json" I0424 15:12:42.790058 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-gvlhl/console-operator_previous.log with fingerprint=15f9d5db047f3c7ccea7aca4bdc2e02e706c03aabfc895c2e83f92c2c230c884 I0424 15:12:42.790221 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-gvlhl/console-operator_current.log with fingerprint=63883782b00698f3922eb82e66b59f77841783b647633c0684d3fde54afbb3a8 I0424 15:12:42.790233 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-7b6d7cfd7f-5vzrt/console_current.log with fingerprint=7f9226d9dd994c7a959dab5e21ef9990238bae045d04c4851687928c674a6178 I0424 15:12:42.790238 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-mxmsx/download-server_current.log with fingerprint=175df4c32693309fde898d3bf054f2a852ad40f6429a408f534a1db8c1590e56 I0424 15:12:42.790333 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-j4qj9 with fingerprint=3e92cdef9a2a774cf73a8820fb90f067028300b93b59b32b294e30345bd7fa79 I0424 15:12:42.790344 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-5868f658b9-wt9qd/registry_current.log with fingerprint=f5f462df48f4f367b2edb2e068c8a79d10e95c634201cedc68462a11539167a8 I0424 15:12:42.790377 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-k8jjt/node-ca_current.log with fingerprint=c683d4d4d51caa625f20f04849640e33740a903aad99bf8518f5046c8177e037 I0424 15:12:42.790384 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-s6xmg/node-ca_current.log with fingerprint=f0b9628d0528f4ec41e269b94186e479764c484acc8c423fc414cfb07d15c31a I0424 15:12:42.790402 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-55b7955667-grf4t/router_current.log with fingerprint=7b236be7909c25b29ef42d8e6020e7c0e36226f9c42d57decde2dc2db49d5c9f I0424 15:12:42.790420 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-774967f6d5-2xt8q/router_current.log with fingerprint=f1d7cac452ef5334bd4093529918597d3e4b947a10f4ba26410324a852612a79 I0424 15:12:42.790477 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-bthf7 with fingerprint=cb18ab179128b9f4d14ca5332a919868c28a241e856603dea59419c24f549904 I0424 15:12:42.790486 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.051452222s to process 23 records I0424 15:12:49.579313 1 tasks_processing.go:74] worker 48 stopped. I0424 15:12:49.579356 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0424 15:12:49.579369 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.840220093s to process 1 records I0424 15:12:50.346028 1 tasks_processing.go:74] worker 46 stopped. I0424 15:12:50.346283 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=2665188add29bb6f18951153176f107c723a75219c598631339e2d0f6e3df1e0 I0424 15:12:50.346298 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.608237258s to process 1 records E0424 15:12:50.346375 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.609s with: function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"silenced_alerts\" failed with an error, function \"active_alerts\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json\"" I0424 15:12:50.347483 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "tsdb_status" failed with an error, function "metrics" failed with an error, function "silenced_alerts" failed with an error, function "active_alerts" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-gvlhl.json" I0424 15:12:50.347500 1 periodic.go:209] Running workloads gatherer I0424 15:12:50.347515 1 tasks_processing.go:45] number of workers: 2 I0424 15:12:50.347523 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 15:12:50.347527 1 tasks_processing.go:71] worker 1 working on workload_info task. I0424 15:12:50.347537 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 15:12:50.347554 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0424 15:12:50.354545 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-207.ec2.internal... I0424 15:12:50.374733 1 tasks_processing.go:74] worker 0 stopped. I0424 15:12:50.374752 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 27.160935ms to process 0 records I0424 15:12:55.328672 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0424 15:12:55.346080 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 21s for image data I0424 15:12:55.354526 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (9ms) I0424 15:12:55.362986 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (8ms) I0424 15:12:55.371722 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0424 15:12:55.380059 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (8ms) I0424 15:12:55.388329 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (8ms) I0424 15:12:55.398557 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (10ms) I0424 15:12:55.407512 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (9ms) I0424 15:12:55.415949 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (8ms) I0424 15:12:55.424257 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (8ms) I0424 15:12:55.434246 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (10ms) I0424 15:12:55.456285 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (22ms) I0424 15:12:55.561205 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (105ms) I0424 15:12:55.656100 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (95ms) I0424 15:12:55.755869 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0424 15:12:55.855692 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0424 15:12:55.956133 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0424 15:12:56.056107 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0424 15:12:56.156272 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0424 15:12:56.255357 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (99ms) I0424 15:12:56.356359 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0424 15:12:56.456755 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0424 15:12:56.555896 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (99ms) I0424 15:12:56.656880 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0424 15:12:56.756261 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0424 15:12:56.856055 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0424 15:12:56.964303 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (108ms) I0424 15:12:57.056513 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (92ms) I0424 15:12:57.156785 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0424 15:12:57.257100 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0424 15:12:57.357046 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0424 15:12:57.456540 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (99ms) I0424 15:12:57.556476 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0424 15:12:57.655498 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0424 15:12:57.756185 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (101ms) I0424 15:12:57.857357 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (101ms) I0424 15:12:57.957063 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0424 15:12:58.055689 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (99ms) I0424 15:12:58.055718 1 tasks_processing.go:74] worker 1 stopped. I0424 15:12:58.055928 1 recorder.go:75] Recording config/workload_info with fingerprint=75a6ce9e82559adcca12354a3eb291e5cded15ece7e838150a8f4706e1f7bc45 I0424 15:12:58.055941 1 gather.go:177] gatherer "workloads" function "workload_info" took 7.708183705s to process 1 records I0424 15:12:58.055955 1 periodic.go:256] Periodic gather workloads completed in 7.708s I0424 15:12:58.055963 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0424 15:12:58.055968 1 periodic.go:209] Running conditional gatherer I0424 15:12:58.061734 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0424 15:12:58.068729 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.21:47414->172.30.0.10:53: read: connection refused I0424 15:12:58.069089 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0424 15:12:58.072534 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:36311->172.30.0.10:53: read: connection refused I0424 15:12:58.072628 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0424 15:12:58.078469 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0424 15:12:58.078485 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078490 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078494 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078497 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078500 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078503 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078506 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078509 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 15:12:58.078512 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0424 15:12:58.078529 1 tasks_processing.go:45] number of workers: 3 I0424 15:12:58.078540 1 tasks_processing.go:69] worker 2 listening for tasks. I0424 15:12:58.078544 1 tasks_processing.go:71] worker 2 working on remote_configuration task. I0424 15:12:58.078550 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 15:12:58.078562 1 tasks_processing.go:71] worker 0 working on rapid_container_logs task. I0424 15:12:58.078565 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 15:12:58.078576 1 tasks_processing.go:74] worker 1 stopped. I0424 15:12:58.078588 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0424 15:12:58.078599 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0424 15:12:58.078610 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 706ns to process 1 records I0424 15:12:58.078675 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0424 15:12:58.078683 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.875µs to process 1 records I0424 15:12:58.078689 1 tasks_processing.go:74] worker 2 stopped. I0424 15:12:58.078811 1 tasks_processing.go:74] worker 0 stopped. I0424 15:12:58.078822 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 241.179µs to process 0 records I0424 15:12:58.078842 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.21:47414->172.30.0.10:53: read: connection refused I0424 15:12:58.078862 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0424 15:12:58.106913 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=a4ae76a25b829a827d1f857951fe9ea30df58acd3d31d090d12c81274e86ccbc I0424 15:12:58.107075 1 diskrecorder.go:70] Writing 124 records to /var/lib/insights-operator/insights-2026-04-24-151258.tar.gz I0424 15:12:58.116767 1 diskrecorder.go:51] Wrote 124 records to disk in 9ms I0424 15:12:58.116800 1 periodic.go:278] Gathering cluster info every 2h0m0s I0424 15:12:58.116815 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0424 15:14:06.736897 1 diskrecorder.go:223] Found files to send: insights-2026-04-24-151258.tar.gz I0424 15:14:06.736933 1 insightsuploader.go:150] Checking archives to upload periodically every 15m4.778025631s I0424 15:14:06.736942 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0424 15:14:06.749110 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0424 15:14:06.983440 1 requests.go:87] Successfully reported id=2026-04-24T15:14:06Z x-rh-insights-request-id=38d5221418754985bc87624b98c0a7db, wrote=69073 I0424 15:14:06.983560 1 insightsuploader.go:187] Uploaded report successfully in 246.608229ms I0424 15:14:06.983585 1 controller.go:128] Initializing last reported time to 2026-04-24T15:14:06Z I0424 15:14:06.983658 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0424 15:14:06.983669 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0424 15:14:06.983679 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0424 15:14:06.989683 1 controller.go:512] The operator is healthy I0424 15:14:36.754300 1 controller.go:512] The operator is healthy I0424 15:15:09.178818 1 insightsreport.go:137] Pulling report from smart-proxy I0424 15:15:09.178844 1 insightsreport.go:149] Retrieving report I0424 15:15:09.187169 1 requests.go:110] Retrieving report for cluster: 732cbcc8-2927-455e-b585-6703246bc32e I0424 15:15:09.187183 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/732cbcc8-2927-455e-b585-6703246bc32e/reports I0424 15:15:09.190078 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/732cbcc8-2927-455e-b585-6703246bc32e/reports I0424 15:15:09.524876 1 insightsreport.go:184] Report retrieved I0424 15:15:09.550804 1 insightsreport.go:239] Report retrieved correctly I0424 15:16:36.758282 1 controller.go:512] The operator is healthy I0424 15:17:36.729788 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0424 15:17:36.735823 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:17:36.735845 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0424 15:17:36.740147 1 secretconfigobserver.go:119] support secret does not exist I0424 15:18:36.753775 1 controller.go:512] The operator is healthy I0424 15:20:36.754928 1 controller.go:512] The operator is healthy I0424 15:22:36.728986 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:22:36.733268 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:22:36.740350 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0424 15:22:36.744640 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:22:36.744655 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0424 15:22:36.748879 1 secretconfigobserver.go:119] support secret does not exist I0424 15:22:36.753898 1 controller.go:512] The operator is healthy I0424 15:24:36.757643 1 controller.go:512] The operator is healthy I0424 15:26:36.789575 1 controller.go:512] The operator is healthy I0424 15:27:36.749849 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0424 15:27:36.754960 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:27:36.754978 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0424 15:27:36.759162 1 secretconfigobserver.go:119] support secret does not exist I0424 15:28:36.756372 1 controller.go:512] The operator is healthy I0424 15:29:16.762470 1 insightsuploader.go:146] Nothing to report since 2026-04-24T15:14:06Z I0424 15:30:36.755844 1 controller.go:512] The operator is healthy I0424 15:32:36.726924 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:32:36.732262 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 15:32:36.754261 1 controller.go:512] The operator is healthy I0424 15:32:36.760082 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0424 15:32:36.765405 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:32:36.765419 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0424 15:32:36.770373 1 secretconfigobserver.go:119] support secret does not exist I0424 15:34:36.753860 1 controller.go:512] The operator is healthy I0424 15:36:36.759911 1 controller.go:512] The operator is healthy I0424 15:37:36.771390 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0424 15:37:36.780660 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 15:37:36.780679 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0424 15:37:36.784923 1 secretconfigobserver.go:119] support secret does not exist I0424 15:38:36.758372 1 controller.go:512] The operator is healthy I0424 15:40:36.754983 1 controller.go:512] The operator is healthy