I0425 19:07:31.582310 1 cmd.go:253] Using service-serving-cert provided certificates I0425 19:07:31.582793 1 observer_polling.go:159] Starting file observer I0425 19:07:32.081163 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0425 19:07:32.081841 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0425 19:07:32.083068 1 secure_serving.go:57] Forcing use of http/1.1 only W0425 19:07:32.083095 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0425 19:07:32.083101 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0425 19:07:32.083105 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0425 19:07:32.083108 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0425 19:07:32.083111 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0425 19:07:32.083113 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0425 19:07:32.083460 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0425 19:07:32.088569 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0425 19:07:32.088574 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0425 19:07:32.088588 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0425 19:07:32.088588 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0425 19:07:32.088602 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0425 19:07:32.088593 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0425 19:07:32.088913 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0425 19:07:32.089151 1 secure_serving.go:213] Serving securely on [::]:8443 I0425 19:07:32.089195 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" I0425 19:07:32.089329 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0425 19:07:32.089388 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"371eac45-4db5-4e2d-942f-5a6da11d0139", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} W0425 19:07:32.093854 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0425 19:07:32.093878 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0425 19:07:32.093980 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0425 19:07:32.101786 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 19:07:32.101804 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0425 19:07:32.105916 1 secretconfigobserver.go:119] support secret does not exist I0425 19:07:32.110232 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 19:07:32.114045 1 secretconfigobserver.go:119] support secret does not exist I0425 19:07:32.117062 1 recorder.go:161] Pruning old reports every 4h38m31s, max age is 288h0m0s I0425 19:07:32.121628 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0425 19:07:32.121649 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0425 19:07:32.121664 1 periodic.go:209] Running clusterconfig gatherer I0425 19:07:32.121662 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0425 19:07:32.121677 1 insightsreport.go:296] Starting report retriever I0425 19:07:32.121685 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0425 19:07:32.121701 1 tasks_processing.go:45] number of workers: 64 I0425 19:07:32.121728 1 tasks_processing.go:69] worker 4 listening for tasks. I0425 19:07:32.121738 1 tasks_processing.go:69] worker 0 listening for tasks. I0425 19:07:32.121747 1 tasks_processing.go:69] worker 1 listening for tasks. I0425 19:07:32.121749 1 tasks_processing.go:71] worker 0 working on install_plans task. I0425 19:07:32.121747 1 tasks_processing.go:69] worker 17 listening for tasks. I0425 19:07:32.121755 1 tasks_processing.go:69] worker 32 listening for tasks. I0425 19:07:32.121760 1 tasks_processing.go:71] worker 17 working on image task. I0425 19:07:32.121761 1 tasks_processing.go:69] worker 2 listening for tasks. I0425 19:07:32.121762 1 tasks_processing.go:69] worker 48 listening for tasks. I0425 19:07:32.121767 1 tasks_processing.go:69] worker 18 listening for tasks. I0425 19:07:32.121769 1 tasks_processing.go:69] worker 33 listening for tasks. I0425 19:07:32.121775 1 tasks_processing.go:69] worker 19 listening for tasks. I0425 19:07:32.121781 1 tasks_processing.go:69] worker 34 listening for tasks. I0425 19:07:32.121785 1 tasks_processing.go:69] worker 20 listening for tasks. I0425 19:07:32.121782 1 tasks_processing.go:69] worker 3 listening for tasks. I0425 19:07:32.121792 1 tasks_processing.go:69] worker 45 listening for tasks. I0425 19:07:32.121782 1 tasks_processing.go:69] worker 41 listening for tasks. I0425 19:07:32.121797 1 tasks_processing.go:69] worker 44 listening for tasks. I0425 19:07:32.121804 1 tasks_processing.go:69] worker 22 listening for tasks. I0425 19:07:32.121806 1 tasks_processing.go:69] worker 37 listening for tasks. I0425 19:07:32.121806 1 tasks_processing.go:69] worker 47 listening for tasks. I0425 19:07:32.121797 1 tasks_processing.go:69] worker 21 listening for tasks. I0425 19:07:32.121816 1 tasks_processing.go:69] worker 23 listening for tasks. I0425 19:07:32.121816 1 tasks_processing.go:69] worker 11 listening for tasks. I0425 19:07:32.121823 1 tasks_processing.go:69] worker 39 listening for tasks. I0425 19:07:32.121827 1 tasks_processing.go:69] worker 24 listening for tasks. I0425 19:07:32.121819 1 tasks_processing.go:69] worker 50 listening for tasks. I0425 19:07:32.121832 1 tasks_processing.go:69] worker 57 listening for tasks. I0425 19:07:32.121833 1 tasks_processing.go:69] worker 43 listening for tasks. I0425 19:07:32.121838 1 tasks_processing.go:69] worker 53 listening for tasks. I0425 19:07:32.121808 1 tasks_processing.go:69] worker 56 listening for tasks. I0425 19:07:32.121839 1 tasks_processing.go:69] worker 40 listening for tasks. I0425 19:07:32.121843 1 tasks_processing.go:69] worker 13 listening for tasks. I0425 19:07:32.121849 1 tasks_processing.go:69] worker 26 listening for tasks. I0425 19:07:32.121843 1 tasks_processing.go:69] worker 9 listening for tasks. I0425 19:07:32.121855 1 tasks_processing.go:69] worker 55 listening for tasks. I0425 19:07:32.121850 1 tasks_processing.go:69] worker 59 listening for tasks. I0425 19:07:32.121856 1 tasks_processing.go:69] worker 5 listening for tasks. I0425 19:07:32.121860 1 tasks_processing.go:69] worker 6 listening for tasks. I0425 19:07:32.121865 1 tasks_processing.go:69] worker 15 listening for tasks. I0425 19:07:32.121865 1 tasks_processing.go:69] worker 16 listening for tasks. I0425 19:07:32.121868 1 tasks_processing.go:69] worker 62 listening for tasks. I0425 19:07:32.121872 1 tasks_processing.go:71] worker 4 working on ingress task. I0425 19:07:32.121871 1 tasks_processing.go:69] worker 61 listening for tasks. I0425 19:07:32.121871 1 tasks_processing.go:69] worker 28 listening for tasks. I0425 19:07:32.121816 1 tasks_processing.go:69] worker 38 listening for tasks. I0425 19:07:32.121797 1 tasks_processing.go:69] worker 36 listening for tasks. I0425 19:07:32.121881 1 tasks_processing.go:69] worker 63 listening for tasks. I0425 19:07:32.121799 1 tasks_processing.go:69] worker 46 listening for tasks. I0425 19:07:32.121789 1 tasks_processing.go:69] worker 35 listening for tasks. I0425 19:07:32.121813 1 tasks_processing.go:69] worker 49 listening for tasks. I0425 19:07:32.121814 1 tasks_processing.go:69] worker 42 listening for tasks. I0425 19:07:32.121826 1 tasks_processing.go:69] worker 60 listening for tasks. I0425 19:07:32.121826 1 tasks_processing.go:69] worker 51 listening for tasks. I0425 19:07:32.121833 1 tasks_processing.go:69] worker 52 listening for tasks. I0425 19:07:32.121826 1 tasks_processing.go:69] worker 14 listening for tasks. I0425 19:07:32.121836 1 tasks_processing.go:69] worker 8 listening for tasks. I0425 19:07:32.121839 1 tasks_processing.go:69] worker 25 listening for tasks. I0425 19:07:32.121841 1 tasks_processing.go:69] worker 58 listening for tasks. I0425 19:07:32.121844 1 tasks_processing.go:69] worker 54 listening for tasks. I0425 19:07:32.121842 1 tasks_processing.go:69] worker 7 listening for tasks. I0425 19:07:32.121852 1 tasks_processing.go:69] worker 10 listening for tasks. I0425 19:07:32.121753 1 tasks_processing.go:71] worker 1 working on lokistack task. I0425 19:07:32.121942 1 tasks_processing.go:71] worker 32 working on infrastructures task. I0425 19:07:32.121948 1 tasks_processing.go:71] worker 24 working on aggregated_monitoring_cr_names task. I0425 19:07:32.121951 1 tasks_processing.go:71] worker 21 working on openstack_dataplanedeployments task. I0425 19:07:32.121957 1 tasks_processing.go:71] worker 44 working on overlapping_namespace_uids task. I0425 19:07:32.121960 1 tasks_processing.go:71] worker 10 working on active_alerts task. I0425 19:07:32.121962 1 tasks_processing.go:71] worker 19 working on openstack_controlplanes task. I0425 19:07:32.121968 1 tasks_processing.go:71] worker 33 working on ceph_cluster task. I0425 19:07:32.121969 1 tasks_processing.go:71] worker 3 working on silenced_alerts task. I0425 19:07:32.121981 1 tasks_processing.go:71] worker 18 working on monitoring_persistent_volumes task. I0425 19:07:32.121999 1 tasks_processing.go:71] worker 11 working on cost_management_metrics_configs task. I0425 19:07:32.122010 1 tasks_processing.go:71] worker 37 working on metrics task. I0425 19:07:32.122011 1 tasks_processing.go:71] worker 39 working on machine_configs task. I0425 19:07:32.121953 1 tasks_processing.go:71] worker 41 working on clusterroles task. I0425 19:07:32.121958 1 tasks_processing.go:71] worker 48 working on ingress_certificates task. I0425 19:07:32.122036 1 tasks_processing.go:71] worker 60 working on nodenetworkconfigurationpolicies task. I0425 19:07:32.121964 1 tasks_processing.go:71] worker 20 working on config_maps task. I0425 19:07:32.122042 1 tasks_processing.go:71] worker 47 working on olm_operators task. I0425 19:07:32.122048 1 tasks_processing.go:71] worker 53 working on cluster_apiserver task. I0425 19:07:32.122052 1 tasks_processing.go:71] worker 23 working on machines task. I0425 19:07:32.122060 1 tasks_processing.go:71] worker 8 working on image_registries task. I0425 19:07:32.121862 1 tasks_processing.go:69] worker 29 listening for tasks. I0425 19:07:32.121869 1 tasks_processing.go:69] worker 27 listening for tasks. I0425 19:07:32.121949 1 tasks_processing.go:71] worker 34 working on machine_healthchecks task. I0425 19:07:32.122070 1 tasks_processing.go:71] worker 29 working on container_images task. I0425 19:07:32.122044 1 tasks_processing.go:71] worker 43 working on openshift_machine_api_events task. I0425 19:07:32.122087 1 tasks_processing.go:71] worker 56 working on machine_config_pools task. I0425 19:07:32.121861 1 tasks_processing.go:69] worker 12 listening for tasks. I0425 19:07:32.122028 1 tasks_processing.go:71] worker 61 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0425 19:07:32.122096 1 tasks_processing.go:71] worker 63 working on openstack_dataplanenodesets task. I0425 19:07:32.122102 1 tasks_processing.go:71] worker 52 working on version task. I0425 19:07:32.122103 1 tasks_processing.go:71] worker 25 working on proxies task. I0425 19:07:32.122109 1 tasks_processing.go:71] worker 42 working on image_pruners task. I0425 19:07:32.122115 1 tasks_processing.go:71] worker 59 working on sap_pods task. I0425 19:07:32.122102 1 tasks_processing.go:71] worker 35 working on oauths task. I0425 19:07:32.122206 1 tasks_processing.go:71] worker 14 working on sap_config task. I0425 19:07:32.122269 1 tasks_processing.go:71] worker 9 working on nodenetworkstates task. I0425 19:07:32.122319 1 tasks_processing.go:71] worker 28 working on networks task. I0425 19:07:32.122357 1 tasks_processing.go:71] worker 40 working on authentication task. I0425 19:07:32.122367 1 tasks_processing.go:71] worker 38 working on support_secret task. I0425 19:07:32.122509 1 tasks_processing.go:71] worker 55 working on pdbs task. I0425 19:07:32.121953 1 tasks_processing.go:71] worker 2 working on openshift_logging task. I0425 19:07:32.122539 1 tasks_processing.go:71] worker 13 working on storage_cluster task. I0425 19:07:32.122038 1 tasks_processing.go:71] worker 57 working on nodes task. I0425 19:07:32.122097 1 tasks_processing.go:71] worker 58 working on openstack_version task. I0425 19:07:32.122336 1 tasks_processing.go:71] worker 5 working on certificate_signing_requests task. I0425 19:07:32.121870 1 tasks_processing.go:69] worker 31 listening for tasks. I0425 19:07:32.122098 1 tasks_processing.go:71] worker 51 working on tsdb_status task. I0425 19:07:32.122917 1 tasks_processing.go:71] worker 31 working on machine_sets task. I0425 19:07:32.121947 1 tasks_processing.go:71] worker 22 working on service_accounts task. I0425 19:07:32.122940 1 tasks_processing.go:71] worker 27 working on node_logs task. I0425 19:07:32.122956 1 tasks_processing.go:71] worker 12 working on mutating_webhook_configurations task. I0425 19:07:32.122105 1 tasks_processing.go:71] worker 49 working on operators task. I0425 19:07:32.122034 1 tasks_processing.go:71] worker 50 working on dvo_metrics task. I0425 19:07:32.122098 1 tasks_processing.go:71] worker 46 working on schedulers task. I0425 19:07:32.121863 1 tasks_processing.go:69] worker 30 listening for tasks. I0425 19:07:32.123324 1 tasks_processing.go:71] worker 30 working on jaegers task. I0425 19:07:32.122345 1 tasks_processing.go:71] worker 15 working on qemu_kubevirt_launcher_logs task. I0425 19:07:32.122349 1 tasks_processing.go:71] worker 16 working on operators_pods_and_events task. I0425 19:07:32.122341 1 tasks_processing.go:71] worker 6 working on validating_webhook_configurations task. I0425 19:07:32.122353 1 tasks_processing.go:71] worker 62 working on machine_autoscalers task. I0425 19:07:32.121955 1 tasks_processing.go:71] worker 45 working on storage_classes task. I0425 19:07:32.122353 1 tasks_processing.go:71] worker 26 working on crds task. I0425 19:07:32.122363 1 tasks_processing.go:71] worker 36 working on sap_datahubs task. I0425 19:07:32.122195 1 tasks_processing.go:71] worker 54 working on pod_network_connectivity_checks task. I0425 19:07:32.122365 1 tasks_processing.go:71] worker 7 working on container_runtime_configs task. E0425 19:07:32.127592 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.204.190:9091: connect: connection refused I0425 19:07:32.127614 1 tasks_processing.go:71] worker 51 working on feature_gates task. E0425 19:07:32.127628 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.204.190:9091: connect: connection refused E0425 19:07:32.127642 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.204.190:9091: connect: connection refused I0425 19:07:32.127653 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.702871ms to process 0 records E0425 19:07:32.127670 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.204.190:9091: connect: connection refused I0425 19:07:32.127688 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 5.643105ms to process 0 records I0425 19:07:32.127701 1 tasks_processing.go:74] worker 37 stopped. I0425 19:07:32.128696 1 tasks_processing.go:74] worker 23 stopped. E0425 19:07:32.128706 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0425 19:07:32.128713 1 gather.go:177] gatherer "clusterconfig" function "machines" took 6.632584ms to process 0 records I0425 19:07:32.136368 1 tasks_processing.go:74] worker 63 stopped. I0425 19:07:32.136382 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 14.259514ms to process 0 records I0425 19:07:32.147559 1 tasks_processing.go:74] worker 1 stopped. I0425 19:07:32.147576 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 25.626418ms to process 0 records I0425 19:07:32.147585 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 25.289976ms to process 0 records I0425 19:07:32.147589 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 25.604268ms to process 0 records I0425 19:07:32.147594 1 tasks_processing.go:74] worker 21 stopped. I0425 19:07:32.147596 1 tasks_processing.go:74] worker 14 stopped. I0425 19:07:32.147599 1 tasks_processing.go:74] worker 34 stopped. E0425 19:07:32.147607 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0425 19:07:32.147619 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 25.521492ms to process 0 records I0425 19:07:32.147714 1 tasks_processing.go:74] worker 9 stopped. I0425 19:07:32.147726 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 25.399702ms to process 0 records I0425 19:07:32.147742 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 25.592907ms to process 0 records I0425 19:07:32.147752 1 tasks_processing.go:74] worker 59 stopped. I0425 19:07:32.147821 1 tasks_processing.go:74] worker 58 stopped. I0425 19:07:32.147831 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 24.973892ms to process 0 records I0425 19:07:32.147850 1 tasks_processing.go:74] worker 31 stopped. I0425 19:07:32.147857 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 24.920657ms to process 0 records I0425 19:07:32.147871 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 25.878626ms to process 0 records I0425 19:07:32.147907 1 tasks_processing.go:74] worker 19 stopped. I0425 19:07:32.147989 1 controller.go:128] Initializing last reported time to 2026-04-25T19:07:27Z I0425 19:07:32.147999 1 tasks_processing.go:74] worker 17 stopped. I0425 19:07:32.148009 1 controller.go:326] The initial operator extension status is healthy I0425 19:07:32.148021 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0425 19:07:32.148027 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0425 19:07:32.148032 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0425 19:07:32.148041 1 recorder.go:75] Recording config/image with fingerprint=9e8198e1f5780c626d355b76858c0a49204062074c58a82cee1552e6602fd40f I0425 19:07:32.148056 1 gather.go:177] gatherer "clusterconfig" function "image" took 26.097053ms to process 1 records I0425 19:07:32.148057 1 controller.go:489] The operator is still being initialized I0425 19:07:32.148072 1 controller.go:512] The operator is healthy I0425 19:07:32.148072 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 25.188904ms to process 0 records I0425 19:07:32.148079 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 25.992349ms to process 0 records I0425 19:07:32.148085 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 25.879483ms to process 0 records I0425 19:07:32.148155 1 tasks_processing.go:74] worker 13 stopped. I0425 19:07:32.148162 1 tasks_processing.go:74] worker 56 stopped. I0425 19:07:32.148156 1 tasks_processing.go:74] worker 33 stopped. I0425 19:07:32.148255 1 tasks_processing.go:74] worker 4 stopped. I0425 19:07:32.148332 1 recorder.go:75] Recording config/ingress with fingerprint=1e10cc54e5fe5119816e177b63400b4442203ef4cdcf466f67bb9500b20b94ed I0425 19:07:32.148349 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 26.155777ms to process 1 records I0425 19:07:32.148429 1 tasks_processing.go:74] worker 32 stopped. I0425 19:07:32.148815 1 recorder.go:75] Recording config/infrastructure with fingerprint=f0cd63ff7dca4fe5cc96bd42170466241366e42b1fa4f97c030b1fcef41343f5 I0425 19:07:32.148826 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 26.107815ms to process 1 records I0425 19:07:32.148915 1 tasks_processing.go:74] worker 40 stopped. I0425 19:07:32.148958 1 recorder.go:75] Recording config/authentication with fingerprint=b0aa895500e030ac506f9e5a0f3788023536f50ed3d7aa8c6758aafe670e5659 I0425 19:07:32.148967 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 25.774895ms to process 1 records I0425 19:07:32.149045 1 tasks_processing.go:74] worker 42 stopped. I0425 19:07:32.149138 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=5e155ab071719a038094960bd0cfc0a3419d578240cfe2f27e691e0635326e74 I0425 19:07:32.149147 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 26.110353ms to process 1 records I0425 19:07:32.149190 1 tasks_processing.go:74] worker 28 stopped. I0425 19:07:32.149296 1 recorder.go:75] Recording config/network with fingerprint=2018fc17dac74bf748e5efdd5003f466df1e932376d5d5bfaa65c08e2f9db464 I0425 19:07:32.149306 1 gather.go:177] gatherer "clusterconfig" function "networks" took 25.862809ms to process 1 records I0425 19:07:32.149391 1 tasks_processing.go:74] worker 35 stopped. I0425 19:07:32.149466 1 recorder.go:75] Recording config/oauth with fingerprint=76cb560c3691e88dcd80e471fe515832bd236d08b37dfb0e946bb2ab09473857 I0425 19:07:32.149474 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 26.030934ms to process 1 records I0425 19:07:32.150654 1 tasks_processing.go:74] worker 54 stopped. E0425 19:07:32.150670 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0425 19:07:32.150681 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 25.722586ms to process 0 records I0425 19:07:32.150728 1 tasks_processing.go:74] worker 25 stopped. I0425 19:07:32.150818 1 recorder.go:75] Recording config/proxy with fingerprint=a8d30b7364d5875d21888571840310d8ca20406699742933544c95d604012c6f I0425 19:07:32.150829 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 28.616104ms to process 1 records I0425 19:07:32.151467 1 tasks_processing.go:74] worker 30 stopped. I0425 19:07:32.151481 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 28.135136ms to process 0 records I0425 19:07:32.152941 1 tasks_processing.go:74] worker 46 stopped. I0425 19:07:32.153023 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=adfe5c4c13400c7fd06e7202a98389f95516a15929436c96c81278165426ad09 I0425 19:07:32.153046 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 29.729682ms to process 1 records I0425 19:07:32.154777 1 tasks_processing.go:74] worker 60 stopped. I0425 19:07:32.154794 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 32.731609ms to process 0 records I0425 19:07:32.154803 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 32.106734ms to process 0 records I0425 19:07:32.154810 1 tasks_processing.go:74] worker 2 stopped. I0425 19:07:32.154855 1 tasks_processing.go:74] worker 38 stopped. E0425 19:07:32.154869 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0425 19:07:32.154876 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 32.473544ms to process 0 records I0425 19:07:32.154960 1 tasks_processing.go:74] worker 62 stopped. I0425 19:07:32.154979 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 30.791589ms to process 0 records I0425 19:07:32.155152 1 tasks_processing.go:74] worker 12 stopped. I0425 19:07:32.155237 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=9be39cd4d31714630ad826e59b65ba7766ec05e03a70926c4a1a1fd40d61ce62 I0425 19:07:32.155288 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=49f854417011cdb7b449663d66a5984ae84e8bcb90a2550e1712b8ec65fe9136 I0425 19:07:32.155601 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=ea1b7d1541563b8f64cb3aa25b46803b7ece1fe9a26e11d6b75cd4152f4354cc I0425 19:07:32.155627 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 31.999166ms to process 3 records I0425 19:07:32.155715 1 tasks_processing.go:74] worker 57 stopped. I0425 19:07:32.156974 1 recorder.go:75] Recording config/node/ip-10-0-0-112.ec2.internal with fingerprint=ace40fea095fe0acd34cd6e6866bf66acb17a5ed2207007071883cae9c87e3e4 I0425 19:07:32.157392 1 recorder.go:75] Recording config/node/ip-10-0-1-240.ec2.internal with fingerprint=dbdf88424c300b94388ae810b679db9978c3e419863fc5a663621990726ef849 I0425 19:07:32.157619 1 recorder.go:75] Recording config/node/ip-10-0-2-198.ec2.internal with fingerprint=baa0f9de047351e41ae3ed8073cf85236394560777e3136f46fad7be1b59ce3a I0425 19:07:32.157939 1 gather_logs.go:145] no pods in namespace were found I0425 19:07:32.157943 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 32.383761ms to process 3 records I0425 19:07:32.157969 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 30.544915ms to process 0 records I0425 19:07:32.158019 1 tasks_processing.go:74] worker 36 stopped. I0425 19:07:32.158042 1 tasks_processing.go:74] worker 55 stopped. I0425 19:07:32.158223 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=a9a67711de587608121b386ce5cfaa7191fb274ca98c0cd3cbd6adce8988a41d I0425 19:07:32.158257 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=fa293c52b026b836814fe2554ad138d07b98100a1a14e5f3d127b8af021d1a60 I0425 19:07:32.158287 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=5a30b5ea51da2887b03195ce772a9a44caeab47528fe1303b2fe11cff8777769 I0425 19:07:32.158316 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=0995e20b3433580cdc65e0ffd5edd153d391ae8b445f1b9563781721fa432672 I0425 19:07:32.158361 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=4588a052c6da6182f6bae1bfcc7c57244ca59df9950ad2d65fc300170a327644 I0425 19:07:32.158395 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=198c76b6387f18f7956e9e9afc3a39d073c60ef276c179d3dbc3a127169703be I0425 19:07:32.158421 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=7329a85ddc9944fd1ba59c6c83681f5727c1db4b009a1fb9ee70da95bb684b62 I0425 19:07:32.158448 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=95c2765412b403989681b15b6ef3647affab0922856aa335ff498b47a159e05b I0425 19:07:32.158479 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=8b1983a880c5784b1c366bf6cbfcd9c20f49ae15dc699510870f952e6f9b26d1 I0425 19:07:32.158519 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=f100e948010214d8b57254d9b8d5ff34a68db67e3a34df5bb3681c8fe2085439 I0425 19:07:32.158558 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=1a036bf98ee9b52a510f578410400838ba67b7dc959588eabec1fa3168d1c91b I0425 19:07:32.158604 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 34.300001ms to process 11 records I0425 19:07:32.158638 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 35.548355ms to process 0 records I0425 19:07:32.158661 1 tasks_processing.go:74] worker 11 stopped. I0425 19:07:32.158767 1 tasks_processing.go:74] worker 53 stopped. I0425 19:07:32.159097 1 recorder.go:75] Recording config/apiserver with fingerprint=073260ac16db6b834a8428106a9e86020f86cb4d3d4a225116a360770a796b03 I0425 19:07:32.159192 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 35.60946ms to process 1 records I0425 19:07:32.159219 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 32.709422ms to process 0 records I0425 19:07:32.159236 1 tasks_processing.go:74] worker 7 stopped. I0425 19:07:32.159252 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0425 19:07:32.159273 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 35.941644ms to process 1 records I0425 19:07:32.159285 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 34.622655ms to process 0 records I0425 19:07:32.159308 1 tasks_processing.go:74] worker 18 stopped. I0425 19:07:32.159308 1 tasks_processing.go:74] worker 44 stopped. I0425 19:07:32.159312 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 36.243214ms to process 0 records I0425 19:07:32.159316 1 tasks_processing.go:74] worker 15 stopped. I0425 19:07:32.159334 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 36.333376ms to process 0 records I0425 19:07:32.159357 1 tasks_processing.go:74] worker 43 stopped. I0425 19:07:32.159759 1 tasks_processing.go:74] worker 45 stopped. I0425 19:07:32.160921 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=906692db69a9084f0eb6e376a3f88a9d4f8b25d7ed9ad1905698a710d4ee1dbf I0425 19:07:32.160964 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=bb4ec426062cfd7148f333318a33fdc2650ae6a1ea9a46b8fda3e4dbd3b35168 I0425 19:07:32.160975 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 35.35604ms to process 2 records I0425 19:07:32.161000 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 37.949419ms to process 0 records I0425 19:07:32.161038 1 tasks_processing.go:74] worker 27 stopped. I0425 19:07:32.161113 1 tasks_processing.go:74] worker 8 stopped. I0425 19:07:32.161654 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=dcda4962ea441f381bc83387213041bc36f780bf960cf06aa90581878d086044 I0425 19:07:32.161672 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 38.905395ms to process 1 records W0425 19:07:32.161686 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0425 19:07:32.162361 1 tasks_processing.go:74] worker 5 stopped. I0425 19:07:32.162379 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 39.482547ms to process 0 records I0425 19:07:32.162915 1 tasks_processing.go:74] worker 51 stopped. I0425 19:07:32.163155 1 recorder.go:75] Recording config/featuregate with fingerprint=cc9e665956a3a09046477d2f1560dc03ee6a5961273649b82a0415e8b465ce32 I0425 19:07:32.163198 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 35.285628ms to process 1 records I0425 19:07:32.172407 1 tasks_processing.go:74] worker 47 stopped. I0425 19:07:32.172526 1 recorder.go:75] Recording config/olm_operators with fingerprint=f56dc234aeb3ec3bcea7ba32384b17be606c99123e31f83ba0de8955d5f89a8f I0425 19:07:32.172544 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 50.352427ms to process 1 records I0425 19:07:32.172664 1 tasks_processing.go:74] worker 6 stopped. I0425 19:07:32.172798 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=e215a4eb0da192149c3011dd3f02976d9d182991d0b5cbc45373c30ca3f7708a I0425 19:07:32.172858 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=a6ce4f232a0b449684930a16e52bf0f262ecb4230be9685fcf1d91bf4fab95a0 I0425 19:07:32.172967 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=33a094f537fa036025d6010f056a949c2899e91cd817e7e34d527f99adbb7b51 I0425 19:07:32.173002 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=c3173517085c26b82fec24faa40a8334d353ea2d7e7c6dae1186ed99b23d5f19 I0425 19:07:32.173037 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=ccffaaace15a0cf4f9041b028ea90e89b08ae98937ea6774493a6fbf6969997c I0425 19:07:32.173075 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=1ea942d45c65afe3ab75bb21e7b612e225a4d4c067c3e9a16879270c80b2b0a1 I0425 19:07:32.173109 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0425 19:07:32.173114 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0425 19:07:32.173114 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=af4d07a10236db3231acd8202ab0401d4d3a4e162c3d78711d9be68a5d98e519 I0425 19:07:32.173247 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=d24b97c41d6170f90ba289588f56e563dd4a7962a64bc79b38cd9ad12f9da33b I0425 19:07:32.173283 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=5ba7858e11254aced3bbf41fd70519c9eafacf9adbb5e1e8431e1c90d0dea1ea I0425 19:07:32.173318 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=83d04acfc31f876324026505a8599b9590239793227a776090b00e5270e75143 W0425 19:07:32.173327 1 operator.go:288] started I0425 19:07:32.173340 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=7ebe8f0d3ecc5647aa7f3640fbb533261f46dfa2623ceb48f38f6872944afc94 I0425 19:07:32.173405 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=baa8df33d7dc7faef684adb4e380303297ed4e520bcfd02cbd86254b2f2a0054 I0425 19:07:32.173341 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0425 19:07:32.173452 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=c208229a9e6da430fcaab084227210483552881ea7483bbe06ef785977c04901 I0425 19:07:32.173466 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 48.770593ms to process 13 records I0425 19:07:32.178817 1 tasks_processing.go:74] worker 16 stopped. I0425 19:07:32.178839 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 55.38672ms to process 0 records I0425 19:07:32.185690 1 tasks_processing.go:74] worker 41 stopped. I0425 19:07:32.185963 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=06ccff8835eb12f5eb9c2e534349189474378828bcb409532254f67150b3bfad I0425 19:07:32.186056 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=7b9fb24d8c46fdf8769626d50e3cac608adbe9dc55d1480a0302e43a7fed75c8 I0425 19:07:32.186065 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 63.65067ms to process 2 records I0425 19:07:32.186336 1 tasks_processing.go:74] worker 26 stopped. I0425 19:07:32.186878 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=09a4a5bcbf8ebba5f25b05ef06594c1134e35d30d99bd4b700c5a6c7fc837dae I0425 19:07:32.187105 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=95ff6d5fca2da346a995ba65a17ea5aad79755c476faa4929697920fc7b714f1 I0425 19:07:32.187116 1 gather.go:177] gatherer "clusterconfig" function "crds" took 61.723354ms to process 2 records I0425 19:07:32.189511 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0425 19:07:32.189575 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0425 19:07:32.189595 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0425 19:07:32.194083 1 base_controller.go:82] Caches are synced for ConfigController I0425 19:07:32.194102 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0425 19:07:32.194709 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0425 19:07:32.194726 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0425 19:07:32.194731 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0425 19:07:32.194738 1 controller.go:212] Source scaController *sca.Controller is not ready I0425 19:07:32.194743 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0425 19:07:32.194770 1 controller.go:489] The operator is still being initialized I0425 19:07:32.194781 1 controller.go:512] The operator is healthy I0425 19:07:32.195241 1 tasks_processing.go:74] worker 24 stopped. I0425 19:07:32.195260 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 73.27729ms to process 0 records I0425 19:07:32.197406 1 tasks_processing.go:74] worker 29 stopped. I0425 19:07:32.198951 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-5mhkv with fingerprint=776cbb7562d077520c681b83fc2148777240d23e8a7b2d13ea557f58e3916dbd I0425 19:07:32.199065 1 recorder.go:75] Recording config/running_containers with fingerprint=c526c665f9078d58033d1f39d2817a1de4b233955c4454d4503bd96791d3d165 I0425 19:07:32.199079 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 75.318703ms to process 2 records I0425 19:07:32.200473 1 tasks_processing.go:74] worker 61 stopped. I0425 19:07:32.200488 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 78.370826ms to process 0 records I0425 19:07:32.200557 1 configmapobserver.go:84] configmaps "insights-config" not found I0425 19:07:32.209243 1 tasks_processing.go:74] worker 48 stopped. E0425 19:07:32.209265 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0425 19:07:32.209274 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ptbrbt7vfkb791ln8lthd9ho69udpo0-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ptbrbt7vfkb791ln8lthd9ho69udpo0-primary-cert-bundle-secret" not found I0425 19:07:32.209381 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=e482c7694af3414a16252eab76cd496807a0eefe908e93d9c618b48a9b074d6d I0425 19:07:32.209399 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 87.190245ms to process 1 records I0425 19:07:32.209942 1 tasks_processing.go:74] worker 20 stopped. E0425 19:07:32.209956 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0425 19:07:32.209963 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0425 19:07:32.209966 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0425 19:07:32.209977 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0425 19:07:32.210008 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0425 19:07:32.210016 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0425 19:07:32.210019 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0425 19:07:32.210043 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0425 19:07:32.210082 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0425 19:07:32.210090 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0425 19:07:32.210095 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 87.886693ms to process 7 records W0425 19:07:32.210831 1 gather_cluster_operators.go:121] Can't read operator resource versions: unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: stale GroupVersion discovery: metrics.k8s.io/v1beta1 I0425 19:07:32.210870 1 tasks_processing.go:74] worker 49 stopped. I0425 19:07:32.210916 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=75e11215beaa88940b35de3f67f8228c3c8c430599b11b8464e2c2e1b4d17e4a I0425 19:07:32.210941 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=4580a7dbead47dd0b3ccfd3d3b89ee5c2cc373ad0472f0ee645053f40a3a57ab I0425 19:07:32.210964 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=01f25511b33dda247cfa244cb3993fc183e1f22336f4dcce613a2b9a2ea5e36a I0425 19:07:32.210988 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=28fd23f2d55c6b262972a88dcb49198f56ecb0b73cb361f5f2ae8c590f6c7118 I0425 19:07:32.211014 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=b6692484d997161dd5144b5a6dbfa72771da9083f98dad07ce2e115a9b176b9a I0425 19:07:32.211051 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=627b9776386d6512f021f9932a63e3d38dbe3df7fed5dc5692b24c2541799ca6 I0425 19:07:32.211070 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=2ab4f6a6df15891e3e90f31e719317c161efc364dd07f713bacb502ffb21a07c I0425 19:07:32.211088 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=8524a735dcea9be0b4a23b0aaaa56e48a0421002a25fde9b252ebe7a9acff33e I0425 19:07:32.211105 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=ee223991bad288fe55f20ad6a444f939c781c2820e395a58b22bd5dd8674b758 I0425 19:07:32.211126 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=28eac1bc939ceffe6624320db3534fa83f464a509378ddf6b16323779ee6229d I0425 19:07:32.211147 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=1bee19880e387c7c0f043f7b3d84c2f54a2631aa5de9206f2165ff1b717fd499 I0425 19:07:32.211282 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=1493bfb0944e88c776660fb0ef0a2f76e1e4f9e676aad4bb360700a090faa1d8 I0425 19:07:32.211303 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=2ed9088e31c78caf616b76cce4570910b493119db5f20e2dd579adc832f732ed I0425 19:07:32.211326 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=e36873cdfceaaa0b9f592ec97a4e6afe0c5af0a2a0c26c0283b945442d6394ec I0425 19:07:32.211343 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=f691f1b6d816c0c0b7514a545857d067049ab7af6d20fa82f621e6f06c45e460 I0425 19:07:32.211361 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=3fb44cef571d8f43603b83fd0f9e2e2e9604faa37a8e64f67cfe0607ff4ed69a I0425 19:07:32.211375 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=b0be849e797a5764f6de8054cd1b093c364883e2686e0f55e258b48393d6488f I0425 19:07:32.211388 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=1a708cce8deafb29217a37dd95fe8bfa7c84c967f0ffc631620e40e74c59e45b I0425 19:07:32.211404 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=e90284716995c0723d735bb236fc7d36951015f70dea0c991f6a1994f0fea3c2 I0425 19:07:32.211425 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=95635f058d3189d50b3c6ec98eec01ed3111b5515009f3570d208761c4d03489 I0425 19:07:32.211449 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=df0a5cf6d610f276d50c11b566ee8ad53bc17c4e2656edadb941120b8485a34d I0425 19:07:32.211457 1 gather.go:177] gatherer "clusterconfig" function "operators" took 87.778853ms to process 21 records I0425 19:07:32.237690 1 tasks_processing.go:74] worker 52 stopped. I0425 19:07:32.238010 1 recorder.go:75] Recording config/version with fingerprint=d4c57fddf76d00f37cad7a3a8f24174179eb990b8ef50684349ceb2a9a2b06c5 I0425 19:07:32.238024 1 recorder.go:75] Recording config/id with fingerprint=cfb25155f610ac20bc818159f653e22fc5f1e814b0970f197623dd30a5fa38ba I0425 19:07:32.238031 1 gather.go:177] gatherer "clusterconfig" function "version" took 115.569572ms to process 2 records I0425 19:07:32.258515 1 tasks_processing.go:74] worker 39 stopped. I0425 19:07:32.258556 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0425 19:07:32.258567 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 136.480823ms to process 1 records I0425 19:07:32.262184 1 cluster_transfer.go:102] no available accepted cluster transfer I0425 19:07:32.262200 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0425 19:07:32.266662 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0425 19:07:32.273783 1 base_controller.go:82] Caches are synced for LoggingSyncer I0425 19:07:32.273800 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0425 19:07:32.439981 1 sca.go:266] etc-pki-entitlement secret successfully created I0425 19:07:32.440023 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated W0425 19:07:33.160684 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0425 19:07:33.183273 1 tasks_processing.go:74] worker 3 stopped. I0425 19:07:33.183296 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0425 19:07:33.183310 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.061280645s to process 1 records I0425 19:07:33.183425 1 tasks_processing.go:74] worker 10 stopped. I0425 19:07:33.183467 1 recorder.go:75] Recording config/alerts with fingerprint=c8a1cf3cce9e3180e913296a8d01195e39bda1fa78e0220a285a73e249c551dc I0425 19:07:33.183480 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.061448701s to process 1 records W0425 19:07:34.160938 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0425 19:07:35.161007 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0425 19:07:36.161139 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0425 19:07:37.159303 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0425 19:07:37.159352 1 tasks_processing.go:74] worker 50 stopped. E0425 19:07:37.159355 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0425 19:07:37.159376 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0425 19:07:37.159387 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.036172978s to process 1 records I0425 19:07:44.965505 1 tasks_processing.go:74] worker 0 stopped. I0425 19:07:44.965555 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0425 19:07:44.965567 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.843738222s to process 1 records I0425 19:07:45.731015 1 tasks_processing.go:74] worker 22 stopped. I0425 19:07:45.731320 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=f5738032f28aea743b389898fadee779c5cd56b79a04dcc121fb2c80f0e9fc94 I0425 19:07:45.731336 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.608069799s to process 1 records E0425 19:07:45.731410 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.609s with: function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0425 19:07:45.732521 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "tsdb_status" failed with an error, function "metrics" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0425 19:07:45.732536 1 periodic.go:209] Running workloads gatherer I0425 19:07:45.732554 1 tasks_processing.go:45] number of workers: 2 I0425 19:07:45.732563 1 tasks_processing.go:69] worker 1 listening for tasks. I0425 19:07:45.732567 1 tasks_processing.go:71] worker 1 working on workload_info task. I0425 19:07:45.732574 1 tasks_processing.go:69] worker 0 listening for tasks. I0425 19:07:45.732591 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0425 19:07:45.739276 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-198.ec2.internal... I0425 19:07:45.739292 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-240.ec2.internal... I0425 19:07:45.739283 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-112.ec2.internal... I0425 19:07:45.766149 1 tasks_processing.go:74] worker 0 stopped. I0425 19:07:45.766184 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 33.534515ms to process 0 records I0425 19:07:51.647481 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 6s I0425 19:07:51.673792 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0425 19:07:51.682088 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0425 19:07:51.690917 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0425 19:07:51.699703 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (9ms) I0425 19:07:51.708438 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (9ms) I0425 19:07:51.717039 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (9ms) I0425 19:07:51.725765 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (9ms) I0425 19:07:51.736294 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (11ms) I0425 19:07:51.746230 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (10ms) I0425 19:07:51.755204 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (9ms) I0425 19:07:51.765445 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (10ms) I0425 19:07:51.782195 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (17ms) I0425 19:07:51.893695 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (111ms) I0425 19:07:51.990111 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (96ms) I0425 19:07:52.088448 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (98ms) I0425 19:07:52.183490 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (95ms) I0425 19:07:52.282243 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (99ms) I0425 19:07:52.383745 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (101ms) I0425 19:07:52.482467 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (99ms) I0425 19:07:52.582499 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0425 19:07:52.682113 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0425 19:07:52.782996 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (101ms) I0425 19:07:52.882882 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (100ms) I0425 19:07:52.982421 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (100ms) I0425 19:07:53.082052 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0425 19:07:53.183030 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (101ms) I0425 19:07:53.283060 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (100ms) I0425 19:07:53.383161 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0425 19:07:53.483504 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0425 19:07:53.582300 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (99ms) I0425 19:07:53.683091 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (101ms) I0425 19:07:53.783003 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (100ms) I0425 19:07:53.883849 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (101ms) I0425 19:07:53.982585 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0425 19:07:54.082006 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (99ms) I0425 19:07:54.182222 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0425 19:07:54.282643 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0425 19:07:54.382221 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (100ms) I0425 19:07:54.482822 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0425 19:07:54.582677 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0425 19:07:54.682471 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0425 19:07:54.783063 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0425 19:07:54.883919 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (101ms) I0425 19:07:54.982969 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (99ms) I0425 19:07:55.083199 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0425 19:07:55.183283 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (100ms) I0425 19:07:55.284243 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (101ms) I0425 19:07:55.383106 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0425 19:07:55.483264 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0425 19:07:55.483296 1 tasks_processing.go:74] worker 1 stopped. E0425 19:07:55.483315 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: received unexpected status code 500 Internal Server Error from https://10.129.0.15:8443/gather_runtime_info I0425 19:07:55.483628 1 recorder.go:75] Recording config/workload_info with fingerprint=e0685be0eed8027ab937ee34330cb599bd82f58f29c2381a630f8b17103372e0 I0425 19:07:55.483645 1 gather.go:177] gatherer "workloads" function "workload_info" took 9.750722517s to process 1 records E0425 19:07:55.483673 1 periodic.go:247] "Unhandled Error" err="workloads failed after 9.751s with: function \"workload_info\" failed with an error" I0425 19:07:55.484773 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0425 19:07:55.484786 1 periodic.go:209] Running conditional gatherer I0425 19:07:55.490731 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0425 19:07:55.597725 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0425 19:07:56.670701 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0425 19:07:56.676582 1 conditional_gatherer.go:392] cluster version is '4.20.8' I0425 19:07:56.676612 1 tasks_processing.go:45] number of workers: 3 I0425 19:07:56.676624 1 tasks_processing.go:69] worker 2 listening for tasks. I0425 19:07:56.676628 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0425 19:07:56.676644 1 tasks_processing.go:69] worker 0 listening for tasks. I0425 19:07:56.676651 1 tasks_processing.go:69] worker 1 listening for tasks. I0425 19:07:56.676659 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0425 19:07:56.676659 1 tasks_processing.go:74] worker 1 stopped. I0425 19:07:56.676673 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0425 19:07:56.676708 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0425 19:07:56.676721 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.007µs to process 1 records I0425 19:07:56.676755 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0425 19:07:56.676762 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.422µs to process 1 records I0425 19:07:56.676767 1 tasks_processing.go:74] worker 0 stopped. I0425 19:07:56.676957 1 tasks_processing.go:74] worker 2 stopped. I0425 19:07:56.676971 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 269.983µs to process 0 records I0425 19:07:56.676989 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0425 19:07:56.677006 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0425 19:07:56.704930 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=f6335f9217c1017afa53d985b27d8427a89a3762440aa080e0f8f3330813eb62 I0425 19:07:56.705067 1 diskrecorder.go:70] Writing 93 records to /var/lib/insights-operator/insights-2026-04-25-190756.tar.gz I0425 19:07:56.711143 1 diskrecorder.go:51] Wrote 93 records to disk in 6ms I0425 19:07:56.711191 1 periodic.go:278] Gathering cluster info every 2h0m0s I0425 19:07:56.711206 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0425 19:09:02.122005 1 diskrecorder.go:223] Found files to send: insights-2026-04-25-190756.tar.gz I0425 19:09:02.122038 1 insightsuploader.go:150] Checking archives to upload periodically every 15m45.834290063s I0425 19:09:02.122047 1 insightsuploader.go:165] Uploading latest report since 2026-04-25T19:07:27Z I0425 19:09:02.132658 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0425 19:09:02.274904 1 requests.go:87] Successfully reported id=2026-04-25T19:09:02Z x-rh-insights-request-id=209908d2d3374cb8bc733ceff167a319, wrote=45435 I0425 19:09:02.275009 1 insightsuploader.go:187] Uploaded report successfully in 152.958211ms I0425 19:09:02.275046 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0425 19:09:02.275061 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0425 19:09:02.275070 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0425 19:09:02.280802 1 controller.go:512] The operator is healthy I0425 19:09:32.182850 1 controller.go:512] The operator is healthy I0425 19:10:07.499221 1 insightsreport.go:137] Pulling report from smart-proxy I0425 19:10:07.499245 1 insightsreport.go:149] Retrieving report I0425 19:10:07.506323 1 requests.go:110] Retrieving report for cluster: f2c13ba7-9866-42e0-b86a-038d1e87daa0 I0425 19:10:07.506337 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/f2c13ba7-9866-42e0-b86a-038d1e87daa0/reports I0425 19:10:07.509252 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/f2c13ba7-9866-42e0-b86a-038d1e87daa0/reports I0425 19:10:07.657691 1 insightsreport.go:184] Report retrieved I0425 19:10:07.681426 1 insightsreport.go:239] Report retrieved correctly I0425 19:11:32.184088 1 controller.go:512] The operator is healthy I0425 19:12:32.114534 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0425 19:12:32.122520 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 19:12:32.122539 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0425 19:12:32.126680 1 secretconfigobserver.go:119] support secret does not exist I0425 19:13:32.180399 1 controller.go:512] The operator is healthy I0425 19:15:32.182605 1 controller.go:512] The operator is healthy I0425 19:17:32.126909 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0425 19:17:32.139607 1 configmapobserver.go:84] configmaps "insights-config" not found I0425 19:17:32.148845 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 19:17:32.148865 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0425 19:17:32.155463 1 configmapobserver.go:84] configmaps "insights-config" not found I0425 19:17:32.175399 1 secretconfigobserver.go:119] support secret does not exist I0425 19:17:32.210956 1 controller.go:512] The operator is healthy I0425 19:19:32.182212 1 controller.go:512] The operator is healthy I0425 19:21:32.183745 1 controller.go:512] The operator is healthy I0425 19:22:32.176113 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0425 19:22:32.184608 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0425 19:22:32.184628 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0425 19:22:32.188794 1 secretconfigobserver.go:119] support secret does not exist