W0330 15:10:14.906984 1 cmd.go:257] Using insecure, self-signed certificates I0330 15:10:15.401775 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0330 15:10:15.402115 1 observer_polling.go:159] Starting file observer I0330 15:10:15.813069 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0330 15:10:15.813260 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0330 15:10:15.813790 1 secure_serving.go:57] Forcing use of http/1.1 only W0330 15:10:15.813819 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0330 15:10:15.813825 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0330 15:10:15.813830 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0330 15:10:15.813834 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0330 15:10:15.813837 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0330 15:10:15.813839 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0330 15:10:15.813847 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0330 15:10:15.817698 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0330 15:10:15.817768 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"43b45c05-b8da-452b-8817-31ea6a202e22", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0330 15:10:15.818744 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0330 15:10:15.818745 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0330 15:10:15.818750 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0330 15:10:15.818760 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0330 15:10:15.818761 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0330 15:10:15.818765 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0330 15:10:15.819041 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-522597531/tls.crt::/tmp/serving-cert-522597531/tls.key" I0330 15:10:15.819298 1 secure_serving.go:213] Serving securely on [::]:8443 I0330 15:10:15.819323 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0330 15:10:15.822766 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0330 15:10:15.822788 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0330 15:10:15.822906 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0330 15:10:15.827178 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0330 15:10:15.827194 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0330 15:10:15.831250 1 secretconfigobserver.go:119] support secret does not exist I0330 15:10:15.835463 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0330 15:10:15.839185 1 secretconfigobserver.go:119] support secret does not exist I0330 15:10:15.841635 1 recorder.go:161] Pruning old reports every 7h54m45s, max age is 288h0m0s I0330 15:10:15.846995 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0330 15:10:15.847019 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0330 15:10:15.847020 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0330 15:10:15.847034 1 insightsreport.go:296] Starting report retriever I0330 15:10:15.847038 1 periodic.go:209] Running clusterconfig gatherer I0330 15:10:15.847049 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0330 15:10:15.847080 1 tasks_processing.go:45] number of workers: 64 I0330 15:10:15.847117 1 tasks_processing.go:69] worker 17 listening for tasks. I0330 15:10:15.847123 1 tasks_processing.go:69] worker 8 listening for tasks. I0330 15:10:15.847128 1 tasks_processing.go:71] worker 17 working on monitoring_persistent_volumes task. I0330 15:10:15.847130 1 tasks_processing.go:69] worker 9 listening for tasks. I0330 15:10:15.847133 1 tasks_processing.go:69] worker 36 listening for tasks. I0330 15:10:15.847141 1 tasks_processing.go:69] worker 14 listening for tasks. I0330 15:10:15.847128 1 tasks_processing.go:71] worker 8 working on machine_healthchecks task. I0330 15:10:15.847147 1 tasks_processing.go:69] worker 13 listening for tasks. I0330 15:10:15.847157 1 tasks_processing.go:69] worker 20 listening for tasks. I0330 15:10:15.847160 1 tasks_processing.go:71] worker 13 working on active_alerts task. I0330 15:10:15.847163 1 tasks_processing.go:69] worker 11 listening for tasks. I0330 15:10:15.847166 1 tasks_processing.go:69] worker 21 listening for tasks. I0330 15:10:15.847174 1 tasks_processing.go:69] worker 2 listening for tasks. I0330 15:10:15.847151 1 tasks_processing.go:69] worker 19 listening for tasks. I0330 15:10:15.847180 1 tasks_processing.go:69] worker 23 listening for tasks. I0330 15:10:15.847183 1 tasks_processing.go:69] worker 10 listening for tasks. I0330 15:10:15.847166 1 tasks_processing.go:69] worker 1 listening for tasks. I0330 15:10:15.847183 1 tasks_processing.go:69] worker 50 listening for tasks. I0330 15:10:15.847188 1 tasks_processing.go:69] worker 3 listening for tasks. I0330 15:10:15.847197 1 tasks_processing.go:69] worker 25 listening for tasks. I0330 15:10:15.847199 1 tasks_processing.go:71] worker 9 working on nodenetworkstates task. I0330 15:10:15.847203 1 tasks_processing.go:69] worker 5 listening for tasks. I0330 15:10:15.847207 1 tasks_processing.go:69] worker 37 listening for tasks. I0330 15:10:15.847202 1 tasks_processing.go:69] worker 15 listening for tasks. I0330 15:10:15.847208 1 tasks_processing.go:69] worker 12 listening for tasks. I0330 15:10:15.847216 1 tasks_processing.go:69] worker 38 listening for tasks. I0330 15:10:15.847216 1 tasks_processing.go:69] worker 6 listening for tasks. I0330 15:10:15.847149 1 tasks_processing.go:71] worker 14 working on service_accounts task. I0330 15:10:15.847222 1 tasks_processing.go:71] worker 36 working on config_maps task. I0330 15:10:15.847218 1 tasks_processing.go:69] worker 30 listening for tasks. I0330 15:10:15.847232 1 tasks_processing.go:71] worker 20 working on infrastructures task. I0330 15:10:15.847235 1 tasks_processing.go:71] worker 30 working on silenced_alerts task. I0330 15:10:15.847240 1 tasks_processing.go:69] worker 52 listening for tasks. I0330 15:10:15.847237 1 tasks_processing.go:69] worker 35 listening for tasks. I0330 15:10:15.847248 1 tasks_processing.go:69] worker 53 listening for tasks. I0330 15:10:15.847246 1 tasks_processing.go:69] worker 33 listening for tasks. I0330 15:10:15.847243 1 tasks_processing.go:69] worker 41 listening for tasks. I0330 15:10:15.847258 1 tasks_processing.go:69] worker 54 listening for tasks. I0330 15:10:15.847157 1 tasks_processing.go:69] worker 0 listening for tasks. I0330 15:10:15.847190 1 tasks_processing.go:69] worker 24 listening for tasks. I0330 15:10:15.847265 1 tasks_processing.go:69] worker 55 listening for tasks. I0330 15:10:15.847195 1 tasks_processing.go:69] worker 4 listening for tasks. I0330 15:10:15.847270 1 tasks_processing.go:69] worker 44 listening for tasks. I0330 15:10:15.847197 1 tasks_processing.go:69] worker 7 listening for tasks. I0330 15:10:15.847276 1 tasks_processing.go:69] worker 58 listening for tasks. W0330 15:10:15.847194 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0330 15:10:15.847277 1 tasks_processing.go:69] worker 59 listening for tasks. I0330 15:10:15.847281 1 tasks_processing.go:69] worker 16 listening for tasks. I0330 15:10:15.847284 1 tasks_processing.go:71] worker 37 working on storage_classes task. I0330 15:10:15.847286 1 tasks_processing.go:69] worker 46 listening for tasks. I0330 15:10:15.847289 1 tasks_processing.go:69] worker 26 listening for tasks. I0330 15:10:15.847173 1 tasks_processing.go:69] worker 22 listening for tasks. I0330 15:10:15.847290 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 117.891µs to process 0 records I0330 15:10:15.847295 1 tasks_processing.go:71] worker 15 working on pdbs task. I0330 15:10:15.847300 1 tasks_processing.go:71] worker 25 working on nodes task. W0330 15:10:15.847299 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0330 15:10:15.847299 1 tasks_processing.go:69] worker 28 listening for tasks. I0330 15:10:15.847308 1 tasks_processing.go:71] worker 19 working on sap_pods task. I0330 15:10:15.847310 1 tasks_processing.go:69] worker 49 listening for tasks. I0330 15:10:15.847307 1 tasks_processing.go:71] worker 10 working on jaegers task. I0330 15:10:15.847319 1 tasks_processing.go:69] worker 27 listening for tasks. I0330 15:10:15.847326 1 tasks_processing.go:69] worker 29 listening for tasks. I0330 15:10:15.847329 1 tasks_processing.go:71] worker 3 working on machine_config_pools task. I0330 15:10:15.847338 1 tasks_processing.go:71] worker 5 working on metrics task. I0330 15:10:15.847301 1 tasks_processing.go:71] worker 2 working on openshift_logging task. I0330 15:10:15.847347 1 tasks_processing.go:69] worker 62 listening for tasks. I0330 15:10:15.847355 1 tasks_processing.go:71] worker 62 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. W0330 15:10:15.847364 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0330 15:10:15.847234 1 tasks_processing.go:69] worker 40 listening for tasks. I0330 15:10:15.847228 1 tasks_processing.go:71] worker 6 working on qemu_kubevirt_launcher_logs task. I0330 15:10:15.847408 1 tasks_processing.go:71] worker 38 working on nodenetworkconfigurationpolicies task. I0330 15:10:15.847471 1 tasks_processing.go:71] worker 35 working on pod_network_connectivity_checks task. I0330 15:10:15.847224 1 tasks_processing.go:69] worker 39 listening for tasks. I0330 15:10:15.847541 1 tasks_processing.go:71] worker 46 working on install_plans task. I0330 15:10:15.847588 1 tasks_processing.go:71] worker 27 working on olm_operators task. I0330 15:10:15.847233 1 tasks_processing.go:69] worker 51 listening for tasks. I0330 15:10:15.847701 1 tasks_processing.go:71] worker 51 working on mutating_webhook_configurations task. I0330 15:10:15.847207 1 tasks_processing.go:69] worker 31 listening for tasks. I0330 15:10:15.847224 1 tasks_processing.go:69] worker 34 listening for tasks. I0330 15:10:15.847295 1 tasks_processing.go:69] worker 47 listening for tasks. I0330 15:10:15.847427 1 tasks_processing.go:69] worker 61 listening for tasks. I0330 15:10:15.847251 1 tasks_processing.go:69] worker 42 listening for tasks. I0330 15:10:15.847262 1 tasks_processing.go:69] worker 43 listening for tasks. I0330 15:10:15.847266 1 tasks_processing.go:69] worker 56 listening for tasks. I0330 15:10:15.847279 1 tasks_processing.go:69] worker 45 listening for tasks. I0330 15:10:15.847284 1 tasks_processing.go:71] worker 50 working on operators_pods_and_events task. I0330 15:10:15.847226 1 tasks_processing.go:69] worker 57 listening for tasks. I0330 15:10:15.847292 1 tasks_processing.go:71] worker 11 working on openstack_version task. I0330 15:10:15.847291 1 tasks_processing.go:71] worker 12 working on container_runtime_configs task. I0330 15:10:15.847676 1 tasks_processing.go:69] worker 60 listening for tasks. I0330 15:10:15.847297 1 tasks_processing.go:71] worker 21 working on aggregated_monitoring_cr_names task. I0330 15:10:15.847214 1 tasks_processing.go:69] worker 32 listening for tasks. I0330 15:10:15.847340 1 tasks_processing.go:71] worker 29 working on image_registries task. I0330 15:10:15.847304 1 tasks_processing.go:71] worker 23 working on lokistack task. I0330 15:10:15.847988 1 tasks_processing.go:71] worker 42 working on ceph_cluster task. I0330 15:10:15.847996 1 tasks_processing.go:71] worker 43 working on image task. I0330 15:10:15.848080 1 tasks_processing.go:71] worker 45 working on version task. I0330 15:10:15.847988 1 tasks_processing.go:71] worker 32 working on sap_datahubs task. I0330 15:10:15.848117 1 tasks_processing.go:71] worker 34 working on machine_autoscalers task. I0330 15:10:15.847314 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 67.36µs to process 0 records I0330 15:10:15.847146 1 tasks_processing.go:69] worker 18 listening for tasks. I0330 15:10:15.848348 1 tasks_processing.go:71] worker 18 working on openstack_controlplanes task. I0330 15:10:15.847449 1 tasks_processing.go:69] worker 63 listening for tasks. I0330 15:10:15.847459 1 tasks_processing.go:71] worker 7 working on oauths task. I0330 15:10:15.848443 1 tasks_processing.go:74] worker 63 stopped. I0330 15:10:15.847466 1 tasks_processing.go:71] worker 52 working on overlapping_namespace_uids task. I0330 15:10:15.847521 1 tasks_processing.go:71] worker 58 working on ingress task. I0330 15:10:15.847528 1 tasks_processing.go:71] worker 13 working on openstack_dataplanenodesets task. I0330 15:10:15.848533 1 tasks_processing.go:71] worker 40 working on feature_gates task. I0330 15:10:15.847534 1 tasks_processing.go:71] worker 59 working on openstack_dataplanedeployments task. I0330 15:10:15.847538 1 tasks_processing.go:71] worker 16 working on cluster_apiserver task. I0330 15:10:15.847546 1 tasks_processing.go:71] worker 0 working on clusterroles task. I0330 15:10:15.847550 1 tasks_processing.go:71] worker 33 working on image_pruners task. I0330 15:10:15.847552 1 tasks_processing.go:71] worker 41 working on validating_webhook_configurations task. I0330 15:10:15.847555 1 tasks_processing.go:71] worker 53 working on support_secret task. I0330 15:10:15.847557 1 tasks_processing.go:71] worker 54 working on authentication task. I0330 15:10:15.847569 1 tasks_processing.go:71] worker 55 working on crds task. I0330 15:10:15.847569 1 tasks_processing.go:71] worker 30 working on ingress_certificates task. I0330 15:10:15.847573 1 tasks_processing.go:71] worker 26 working on machine_configs task. I0330 15:10:15.847572 1 tasks_processing.go:71] worker 24 working on container_images task. I0330 15:10:15.847576 1 tasks_processing.go:71] worker 22 working on node_logs task. I0330 15:10:15.847579 1 tasks_processing.go:71] worker 4 working on operators task. I0330 15:10:15.847582 1 tasks_processing.go:71] worker 49 working on machine_sets task. I0330 15:10:15.847585 1 tasks_processing.go:71] worker 28 working on cost_management_metrics_configs task. I0330 15:10:15.847592 1 tasks_processing.go:71] worker 44 working on tsdb_status task. W0330 15:10:15.849500 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0330 15:10:15.847685 1 tasks_processing.go:71] worker 39 working on networks task. I0330 15:10:15.847302 1 tasks_processing.go:69] worker 48 listening for tasks. I0330 15:10:15.849540 1 tasks_processing.go:74] worker 48 stopped. I0330 15:10:15.848123 1 tasks_processing.go:71] worker 31 working on sap_config task. I0330 15:10:15.848124 1 tasks_processing.go:71] worker 56 working on proxies task. I0330 15:10:15.847311 1 tasks_processing.go:71] worker 1 working on dvo_metrics task. I0330 15:10:15.848129 1 tasks_processing.go:71] worker 57 working on storage_cluster task. I0330 15:10:15.848133 1 tasks_processing.go:71] worker 47 working on openshift_machine_api_events task. I0330 15:10:15.848134 1 tasks_processing.go:71] worker 61 working on schedulers task. I0330 15:10:15.848136 1 tasks_processing.go:71] worker 60 working on certificate_signing_requests task. I0330 15:10:15.848332 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.248µs to process 0 records I0330 15:10:15.849843 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 24.593µs to process 0 records I0330 15:10:15.849855 1 tasks_processing.go:74] worker 44 stopped. I0330 15:10:15.848338 1 tasks_processing.go:71] worker 5 working on machines task. I0330 15:10:15.852198 1 tasks_processing.go:74] worker 8 stopped. E0330 15:10:15.852209 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0330 15:10:15.852218 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 5.038999ms to process 0 records I0330 15:10:15.852589 1 tasks_processing.go:74] worker 34 stopped. I0330 15:10:15.852602 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 4.28846ms to process 0 records I0330 15:10:15.853752 1 tasks_processing.go:74] worker 2 stopped. I0330 15:10:15.853761 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 6.401586ms to process 0 records I0330 15:10:15.859396 1 tasks_processing.go:74] worker 37 stopped. I0330 15:10:15.859648 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0330 15:10:15.859676 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0330 15:10:15.859685 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0330 15:10:15.859691 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0330 15:10:15.859717 1 controller.go:489] The operator is still being initialized I0330 15:10:15.859730 1 controller.go:512] The operator is healthy I0330 15:10:15.859795 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=e03af25c0a035cf75dff03700ee5f76c7217284bde5aeb81e2a09b0c6caebc33 I0330 15:10:15.859840 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=74d86df236078ebbae4fcce7c69f47fba49cadbad0c498ac0b8f4c73a5ba8b9e I0330 15:10:15.859847 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 12.103415ms to process 2 records I0330 15:10:15.859979 1 tasks_processing.go:74] worker 20 stopped. I0330 15:10:15.862678 1 recorder.go:75] Recording config/infrastructure with fingerprint=33db57d4a1f23926bb2c37797f13ad2c4d7be7f66e2e869733af3b6513928193 I0330 15:10:15.862696 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 12.550054ms to process 1 records I0330 15:10:15.862776 1 tasks_processing.go:74] worker 7 stopped. I0330 15:10:15.862900 1 recorder.go:75] Recording config/oauth with fingerprint=6e30fb6bc39bb52aa857c905753d4a2edf8e79ca6c0cc139c7476a9d33bb3975 I0330 15:10:15.862910 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 13.002119ms to process 1 records I0330 15:10:15.862999 1 tasks_processing.go:74] worker 25 stopped. I0330 15:10:15.863149 1 recorder.go:75] Recording config/node/ip-10-0-0-16.ec2.internal with fingerprint=95ff6f95fe981640cf796f56479e2cfca2771349803cbcd792c38c7a8805b317 I0330 15:10:15.863202 1 recorder.go:75] Recording config/node/ip-10-0-1-70.ec2.internal with fingerprint=d2f95c0b8fd0ab2c4870d668cf0a14a296615ec6b90f28a110979173e8346d93 I0330 15:10:15.863260 1 recorder.go:75] Recording config/node/ip-10-0-2-205.ec2.internal with fingerprint=f88266a3919ed5835c8c7d983c7bdea2fda21d06999fcbe9b4e64548bd2d0c9c I0330 15:10:15.863266 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 15.262445ms to process 3 records I0330 15:10:15.872389 1 tasks_processing.go:74] worker 18 stopped. I0330 15:10:15.872406 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 24.029095ms to process 0 records I0330 15:10:15.872418 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 25.074271ms to process 0 records I0330 15:10:15.872423 1 tasks_processing.go:74] worker 10 stopped. I0330 15:10:15.872691 1 tasks_processing.go:74] worker 9 stopped. I0330 15:10:15.872706 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 25.482646ms to process 0 records I0330 15:10:15.872754 1 tasks_processing.go:74] worker 35 stopped. E0330 15:10:15.872764 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0330 15:10:15.872770 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 25.266886ms to process 0 records I0330 15:10:15.872906 1 tasks_processing.go:74] worker 38 stopped. I0330 15:10:15.872917 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 25.433083ms to process 0 records I0330 15:10:15.872929 1 tasks_processing.go:74] worker 13 stopped. I0330 15:10:15.872934 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 24.386905ms to process 0 records I0330 15:10:15.872959 1 tasks_processing.go:74] worker 12 stopped. I0330 15:10:15.872979 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 25.081629ms to process 0 records I0330 15:10:15.873072 1 tasks_processing.go:74] worker 16 stopped. I0330 15:10:15.873151 1 recorder.go:75] Recording config/apiserver with fingerprint=47c2010dfef89694b47deb311d2b093e27fc83587652cd19022f5d854820d874 I0330 15:10:15.873170 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 24.276983ms to process 1 records I0330 15:10:15.873187 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 25.184815ms to process 0 records I0330 15:10:15.873196 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 24.915155ms to process 0 records I0330 15:10:15.873204 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 25.150249ms to process 0 records I0330 15:10:15.873213 1 tasks_processing.go:74] worker 23 stopped. I0330 15:10:15.873220 1 tasks_processing.go:74] worker 32 stopped. I0330 15:10:15.873222 1 tasks_processing.go:74] worker 11 stopped. I0330 15:10:15.873300 1 tasks_processing.go:74] worker 42 stopped. I0330 15:10:15.873311 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 25.300626ms to process 0 records I0330 15:10:15.873319 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 26.181652ms to process 0 records I0330 15:10:15.873325 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 24.649567ms to process 0 records I0330 15:10:15.873333 1 tasks_processing.go:74] worker 59 stopped. I0330 15:10:15.873339 1 tasks_processing.go:74] worker 17 stopped. I0330 15:10:15.873396 1 tasks_processing.go:74] worker 58 stopped. I0330 15:10:15.873622 1 recorder.go:75] Recording config/ingress with fingerprint=3a90b11ee4e8a75611c556dab2c0aefe283bc63ffa55c38c2127b3e16a065fd9 I0330 15:10:15.873642 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 24.858518ms to process 1 records I0330 15:10:15.873667 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0330 15:10:15.873678 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 25.048524ms to process 1 records I0330 15:10:15.873730 1 tasks_processing.go:74] worker 52 stopped. I0330 15:10:15.873814 1 tasks_processing.go:74] worker 40 stopped. I0330 15:10:15.873824 1 recorder.go:75] Recording config/featuregate with fingerprint=bf5658aec4022ed684c9f833b5660e17375f7f3fe648aa5b207468ffc51bd6e9 I0330 15:10:15.873834 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 24.973626ms to process 1 records I0330 15:10:15.874573 1 tasks_processing.go:74] worker 19 stopped. I0330 15:10:15.874585 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 27.255841ms to process 0 records I0330 15:10:15.874706 1 tasks_processing.go:74] worker 43 stopped. I0330 15:10:15.874822 1 recorder.go:75] Recording config/image with fingerprint=e3aa62a0ed90be03beb9229ecf3b5eb93412ef7e9282430a2af671209451b399 I0330 15:10:15.874839 1 gather.go:177] gatherer "clusterconfig" function "image" took 26.689188ms to process 1 records I0330 15:10:15.875076 1 tasks_processing.go:74] worker 33 stopped. I0330 15:10:15.875335 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=446838b5a11a47f2b1fd5d10779672208d68d29678d4612d496c5b07a34b9c3d I0330 15:10:15.875349 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 26.242565ms to process 1 records I0330 15:10:15.878206 1 tasks_processing.go:74] worker 29 stopped. I0330 15:10:15.878568 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=db14bd7372a0ba767b3f5efb68c2cceccd2245815bfae07453087c0b2b76a60c I0330 15:10:15.878585 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 30.25265ms to process 1 records I0330 15:10:15.895973 1 tasks_processing.go:74] worker 51 stopped. I0330 15:10:15.896250 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=21c1eeba4ad4e2be7a3679869ad71bffd2e8885bad57eee23d913007955d9f0a I0330 15:10:15.896304 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=755e3801c43fb0b4c603849ef30cf6d180dc4c52daf19ec2b65fbac0efa1c9cf I0330 15:10:15.896355 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=4ca7f2bd10cd702773640221b9327e9ad9ed0da2b1eb4e45fbff41ac4f735e92 I0330 15:10:15.896367 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 48.246877ms to process 3 records I0330 15:10:15.896452 1 tasks_processing.go:74] worker 15 stopped. I0330 15:10:15.896474 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=d2e0f4aacadbd24914ebbf48b667ee345a3b800f543c078c0dcfb9f0992ba634 I0330 15:10:15.896501 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=b90f636c4a181f2b27b9ae1cfaddc6ff61239560fd1ca5d9d9d5e9f444ae8ce2 I0330 15:10:15.896527 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=9772cb9b8847a8f7a3e9df03325f3d3c89e053e087cb6d39df2989dfce6ba4a0 I0330 15:10:15.896538 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 48.686496ms to process 3 records I0330 15:10:15.907158 1 tasks_processing.go:74] worker 54 stopped. I0330 15:10:15.907605 1 recorder.go:75] Recording config/authentication with fingerprint=3329e63ab394848a9850494c6fe77e1d98bbd2ebae139502659bc29727f017a4 I0330 15:10:15.907625 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 58.090464ms to process 1 records I0330 15:10:15.910467 1 tasks_processing.go:74] worker 31 stopped. I0330 15:10:15.910480 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 60.912742ms to process 0 records I0330 15:10:15.910488 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 61.008039ms to process 0 records I0330 15:10:15.910493 1 tasks_processing.go:74] worker 28 stopped. I0330 15:10:15.910522 1 tasks_processing.go:74] worker 62 stopped. I0330 15:10:15.910542 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 63.135894ms to process 0 records I0330 15:10:15.910556 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 60.874487ms to process 0 records I0330 15:10:15.910563 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 61.170463ms to process 0 records E0330 15:10:15.910571 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0330 15:10:15.910579 1 gather.go:177] gatherer "clusterconfig" function "machines" took 60.700796ms to process 0 records I0330 15:10:15.910577 1 tasks_processing.go:74] worker 57 stopped. I0330 15:10:15.910588 1 tasks_processing.go:74] worker 5 stopped. I0330 15:10:15.910589 1 tasks_processing.go:74] worker 49 stopped. I0330 15:10:15.919700 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0330 15:10:15.919715 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0330 15:10:15.919766 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0330 15:10:15.922109 1 tasks_processing.go:74] worker 27 stopped. I0330 15:10:15.922183 1 recorder.go:75] Recording config/olm_operators with fingerprint=4ac2901ffe4349c8895ce3604810aad49c54a51c3cc3c60f9538514284d2fc09 I0330 15:10:15.922199 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 74.475327ms to process 1 records I0330 15:10:15.922225 1 tasks_processing.go:74] worker 0 stopped. I0330 15:10:15.922514 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=e775eaeb6e3f141110d81bff80d26c40ee48cc481c63fbb5652efad06edbc094 I0330 15:10:15.922674 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=65cef6cbc2b2f4a756162a586f2ce1d6c6e4f013344f63d83c2845b7ceb32c22 I0330 15:10:15.922691 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 73.496193ms to process 2 records I0330 15:10:15.922781 1 tasks_processing.go:74] worker 41 stopped. I0330 15:10:15.922828 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0330 15:10:15.922871 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=3fdb06b11ae18bd7a7f6cf2bb3f67abc9f040ae0ba7827d3cff9638bfc3fe45c I0330 15:10:15.922828 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0330 15:10:15.922986 1 operator.go:288] started I0330 15:10:15.923013 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0330 15:10:15.923014 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=868ee6e87fcb38310bb154aaf6c7b6765b6143af5e6a66529e8f9b59bc9594e0 I0330 15:10:15.923061 1 base_controller.go:82] Caches are synced for ConfigController I0330 15:10:15.923068 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=b6a8480743d039c298ce4b19edd9d6caa13616db29b4da2d821504107dc20838 I0330 15:10:15.923075 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0330 15:10:15.923111 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=e007c72650d2f7429255e9612d92a575c808d48bc64fcf1c329ac1d07de58e3e I0330 15:10:15.923139 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=182ad3da6452a2fff42dfb43859ff263d785fc6f51733c924631108477b51aec I0330 15:10:15.923172 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=f0e0b460e9766ca961b81ebe6e046181c3e510dcc589bfb17320deea7601ff61 I0330 15:10:15.923204 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=6a66d9b9d3062c6002883215385ce80897fe136b0e27565af081029b60e8e148 I0330 15:10:15.923247 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=0df5f08fbb7ba6ee0dfe57ab754731fbf850f68b4db825dbdaac51f19d64ad45 I0330 15:10:15.923286 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=2bff8ce6ec48cc90a8c8d1b72ab913d4ac344df13d489238e129432945929e57 I0330 15:10:15.923321 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=3305994088e4be05d43dfaf83aca957b8cc10106b10755be268c596dc2486173 I0330 15:10:15.923358 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=b3fcb358a3ebd3da6f24790eaf04a4d3b7294be41e8b9a31c52d48706c30add2 I0330 15:10:15.923367 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 73.309817ms to process 11 records E0330 15:10:15.923379 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0330 15:10:15.923388 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 73.392547ms to process 0 records I0330 15:10:15.923392 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 72.763596ms to process 0 records I0330 15:10:15.923405 1 tasks_processing.go:74] worker 53 stopped. I0330 15:10:15.923413 1 tasks_processing.go:74] worker 47 stopped. I0330 15:10:15.923449 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=8db4b5125c12ef55de583124c04ca2161c1478721ee5e0a4c04256f36a28c567 I0330 15:10:15.923456 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 72.801795ms to process 1 records I0330 15:10:15.923484 1 tasks_processing.go:74] worker 61 stopped. I0330 15:10:15.923499 1 recorder.go:75] Recording config/proxy with fingerprint=f7d669840f179fdd42858d5eecb96e40cb4bfa1138ca6fe32d210878fb836e44 I0330 15:10:15.923504 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 73.040941ms to process 1 records I0330 15:10:15.923538 1 tasks_processing.go:74] worker 56 stopped. I0330 15:10:15.923604 1 recorder.go:75] Recording config/network with fingerprint=c8b4ab745aaa54915507bdaf4134a26ae7f7e41eecd5058e366dedbe072a8602 I0330 15:10:15.923606 1 tasks_processing.go:74] worker 39 stopped. I0330 15:10:15.923613 1 gather.go:177] gatherer "clusterconfig" function "networks" took 73.13946ms to process 1 records I0330 15:10:15.924122 1 tasks_processing.go:74] worker 60 stopped. I0330 15:10:15.924135 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 74.291149ms to process 0 records W0330 15:10:15.925515 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0330 15:10:15.949170 1 configmapobserver.go:84] configmaps "insights-config" not found I0330 15:10:15.949313 1 tasks_processing.go:74] worker 55 stopped. I0330 15:10:15.949658 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0330 15:10:15.949672 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0330 15:10:15.949699 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0330 15:10:15.949706 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0330 15:10:15.949711 1 controller.go:212] Source scaController *sca.Controller is not ready I0330 15:10:15.949736 1 controller.go:489] The operator is still being initialized I0330 15:10:15.949747 1 controller.go:512] The operator is healthy I0330 15:10:15.950126 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=4c662c81bf76deb3694bc52c3888714a1e3acdfe68e160e0e55a5d837d7fcdf0 I0330 15:10:15.950891 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=cfcc3637580b1436b6365949afcabdb69aa4787016ad16246f52ae8c725ae7f4 I0330 15:10:15.950973 1 gather.go:177] gatherer "clusterconfig" function "crds" took 100.230113ms to process 2 records I0330 15:10:15.955003 1 prometheus_rules.go:88] Prometheus rules successfully created I0330 15:10:15.958464 1 tasks_processing.go:74] worker 22 stopped. I0330 15:10:15.958485 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 109.144718ms to process 0 records E0330 15:10:15.958512 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%278aa0bc5a-5e81-4f4d-b60f-5de2f984448e%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:59169->172.30.0.10:53: read: connection refused I0330 15:10:15.958523 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%278aa0bc5a-5e81-4f4d-b60f-5de2f984448e%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:59169->172.30.0.10:53: read: connection refused I0330 15:10:15.963203 1 tasks_processing.go:74] worker 36 stopped. E0330 15:10:15.963217 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0330 15:10:15.963223 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0330 15:10:15.963227 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0330 15:10:15.963237 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0330 15:10:15.963262 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0330 15:10:15.963270 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0330 15:10:15.963274 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0330 15:10:15.963278 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0330 15:10:15.963316 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0330 15:10:15.963323 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0330 15:10:15.963328 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 115.965239ms to process 7 records I0330 15:10:15.965661 1 tasks_processing.go:74] worker 45 stopped. I0330 15:10:15.965951 1 recorder.go:75] Recording config/version with fingerprint=5838f57e802abb17e7dff180aee1641865f60d66930624ea177681cb10ec64c9 I0330 15:10:15.965971 1 recorder.go:75] Recording config/id with fingerprint=3dd2388e93fcbcf4f5a2eb31c24bbf700e23fc36b34162684cba1c348742be37 I0330 15:10:15.965979 1 gather.go:177] gatherer "clusterconfig" function "version" took 117.503155ms to process 2 records I0330 15:10:15.966035 1 gather_logs.go:145] no pods in namespace were found I0330 15:10:15.966047 1 tasks_processing.go:74] worker 6 stopped. I0330 15:10:15.966054 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 118.662715ms to process 0 records I0330 15:10:15.966859 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0330 15:10:15.970116 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:47164->172.30.0.10:53: read: connection refused I0330 15:10:15.970138 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.12:47164->172.30.0.10:53: read: connection refused I0330 15:10:15.981402 1 tasks_processing.go:74] worker 24 stopped. I0330 15:10:15.982709 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-ptf2j with fingerprint=f32bc73ab731bc44605722956b186642f9094310839be8be1c67e982e7baeb0f I0330 15:10:15.982764 1 recorder.go:75] Recording config/running_containers with fingerprint=fa5cbc5477f1d25296545ab15046fed45effbe33bcb8edfde63365c982a748b9 I0330 15:10:15.982774 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 132.156916ms to process 2 records I0330 15:10:15.994011 1 tasks_processing.go:74] worker 21 stopped. I0330 15:10:15.994029 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 146.105897ms to process 0 records I0330 15:10:16.024095 1 base_controller.go:82] Caches are synced for LoggingSyncer I0330 15:10:16.024111 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0330 15:10:16.053210 1 tasks_processing.go:74] worker 30 stopped. E0330 15:10:16.053231 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0330 15:10:16.053238 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pc3u5acg13brr2ait49j6ehs236k2o5-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pc3u5acg13brr2ait49j6ehs236k2o5-primary-cert-bundle-secret" not found I0330 15:10:16.053309 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=ccf9b58c02c7eca8e782dca9bda57c3886d1a874b64c0b9d481240eea1907702 I0330 15:10:16.053323 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 204.055991ms to process 1 records I0330 15:10:16.062069 1 tasks_processing.go:74] worker 26 stopped. I0330 15:10:16.062124 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0330 15:10:16.062144 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 212.895857ms to process 1 records I0330 15:10:16.313978 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0330 15:10:16.313998 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0330 15:10:16.314232 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-8mtnv pod in namespace openshift-dns (previous: false). I0330 15:10:16.540453 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-8mtnv pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-8mtnv\" is waiting to start: ContainerCreating" I0330 15:10:16.540472 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-8mtnv\" is waiting to start: ContainerCreating" I0330 15:10:16.540481 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-8mtnv pod in namespace openshift-dns (previous: false). I0330 15:10:16.718428 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-8mtnv pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-8mtnv\" is waiting to start: ContainerCreating" I0330 15:10:16.718447 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-8mtnv\" is waiting to start: ContainerCreating" I0330 15:10:16.718479 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-sbhtq pod in namespace openshift-dns (previous: false). I0330 15:10:16.901979 1 tasks_processing.go:74] worker 3 stopped. I0330 15:10:16.901997 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 1.054634657s to process 0 records W0330 15:10:16.925660 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0330 15:10:16.937570 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-sbhtq pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-sbhtq\" is waiting to start: ContainerCreating" I0330 15:10:16.937586 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-sbhtq\" is waiting to start: ContainerCreating" I0330 15:10:16.937594 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-sbhtq pod in namespace openshift-dns (previous: false). I0330 15:10:17.119482 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-sbhtq pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-sbhtq\" is waiting to start: ContainerCreating" I0330 15:10:17.119502 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-sbhtq\" is waiting to start: ContainerCreating" I0330 15:10:17.119530 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-td6nn pod in namespace openshift-dns (previous: false). I0330 15:10:17.344771 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-td6nn pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-td6nn\" is waiting to start: ContainerCreating" I0330 15:10:17.344788 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-td6nn\" is waiting to start: ContainerCreating" I0330 15:10:17.344819 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-td6nn pod in namespace openshift-dns (previous: false). I0330 15:10:17.361220 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0330 15:10:17.519910 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-td6nn pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-td6nn\" is waiting to start: ContainerCreating" I0330 15:10:17.519930 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-td6nn\" is waiting to start: ContainerCreating" I0330 15:10:17.519940 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-d689g pod in namespace openshift-dns (previous: false). I0330 15:10:17.718851 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0330 15:10:17.718870 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-j2kwk pod in namespace openshift-dns (previous: false). I0330 15:10:17.749632 1 tasks_processing.go:74] worker 4 stopped. I0330 15:10:17.749681 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=f34bb7114486aaafd8d4cec51590108b0b3c3f603c95f64fcbf9ac4bcf826807 I0330 15:10:17.749712 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=4abeda629d8e8d36f5deba55b72f90e17e8107ba806c8029fe2951a17e777b71 I0330 15:10:17.749739 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0330 15:10:17.749765 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=83a2211887676d0abbe173d79fa05ac909e509840652546e67f56e44533d265f I0330 15:10:17.749821 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0330 15:10:17.749845 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=428ffb15a425ff8925f066cc3d99137220bd77f80c9ac00221cfee1f4c28665d I0330 15:10:17.749874 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=57d0c8c6352f38311838b1cf85f414d8b282f7f1d837bef3993d04cf0010a435 I0330 15:10:17.749898 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=760408971649507b9f8235823c7551567b828c09a49265c3c0328f53246f5ea5 I0330 15:10:17.749912 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=25768d0b636431099a61c80c510af7b21848ffa610c2669f4524d0a36253e41f I0330 15:10:17.749929 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=bb4eeb938bde475577640691d390e6db2a984d9092e24338001a41d892ae4a8f I0330 15:10:17.749938 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0330 15:10:17.749967 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=779a509938ac225ffa799abe370304720dea4bb23fd6fc49cdefb16010316538 I0330 15:10:17.749978 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0330 15:10:17.749993 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=910a8a5294775673b01e429930c912f79f7a27703b4ae5b6f37c38294111bf4b I0330 15:10:17.750002 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0330 15:10:17.750014 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=d334fd5c15e2cc4a00fafef8b049123f0cb7bf3724756675308227549af0efd9 I0330 15:10:17.750022 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0330 15:10:17.750038 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=755fa31c2e1ebfefe4633152e1071a9115b1e9c90de2a20145bca1ba2f34ff5e I0330 15:10:17.750159 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=8a6f16621d4a2290d4209d3d5ed42102941f66356ef78b44f763fcfa11cd7665 I0330 15:10:17.750168 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0330 15:10:17.750176 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0330 15:10:17.750197 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0330 15:10:17.750217 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=e9718e6dba71969b6b20562e39c88933f3b35186ecfe92df08ad4b68b30850dd I0330 15:10:17.750240 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=b5dc08b867f7fd16510a68de05e9d9d8b7257334857692585f120e3d65da798e I0330 15:10:17.750248 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0330 15:10:17.750264 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=06382069388831c8db7d48f1c816ea878528c70c9dffd81c5fb4c16c2dbc0ae7 I0330 15:10:17.750274 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0330 15:10:17.750286 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=6ae933e734c52148f93b21836fae6432d6ae55bbad554fc109476d99cbd587a3 I0330 15:10:17.750302 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=7c14a27254cd57f04343d8eec2c65611b0ce5f7174583bf2040ed78116dacc69 I0330 15:10:17.750316 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=90f7d5147bce8c6203ca5b2a793c3dea325bcd2cc7d6f8895ac7d1e79a1627bd I0330 15:10:17.750333 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=c6458528c510eac84a165f68833b7307fec827138566986089a12293e73c2c29 I0330 15:10:17.750346 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=f7a7ffccf421e4edd3947cbb04d4fad246c221d90dc0be8e2728cadea063fa59 I0330 15:10:17.750371 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=a48353da6fc68cb02ffda721122b2ca69365ea586665372ac162b73027f748cf I0330 15:10:17.750388 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0330 15:10:17.750398 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0330 15:10:17.750404 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.900294928s to process 35 records I0330 15:10:17.917911 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0330 15:10:17.917930 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-x6fdl pod in namespace openshift-dns (previous: false). W0330 15:10:17.925295 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0330 15:10:18.119438 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0330 15:10:18.119487 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7879584b45-9gg24 pod in namespace openshift-image-registry (previous: false). I0330 15:10:18.323225 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7879584b45-9gg24 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7879584b45-9gg24\" is waiting to start: ContainerCreating" I0330 15:10:18.323247 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7879584b45-9gg24\" is waiting to start: ContainerCreating" I0330 15:10:18.323283 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-8f55956f4-9z5w4 pod in namespace openshift-image-registry (previous: false). I0330 15:10:18.517399 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-8f55956f4-9z5w4 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-8f55956f4-9z5w4\" is waiting to start: ContainerCreating" I0330 15:10:18.517416 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-8f55956f4-9z5w4\" is waiting to start: ContainerCreating" I0330 15:10:18.517457 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-8f55956f4-rshfr pod in namespace openshift-image-registry (previous: false). I0330 15:10:18.718239 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-8f55956f4-rshfr pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-8f55956f4-rshfr\" is waiting to start: ContainerCreating" I0330 15:10:18.718262 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-8f55956f4-rshfr\" is waiting to start: ContainerCreating" I0330 15:10:18.718273 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-6s4q6 pod in namespace openshift-image-registry (previous: false). I0330 15:10:18.922345 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0330 15:10:18.922363 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-hv6mc pod in namespace openshift-image-registry (previous: false). W0330 15:10:18.926379 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0330 15:10:19.121699 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0330 15:10:19.121722 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-z5xhg pod in namespace openshift-image-registry (previous: false). I0330 15:10:19.318772 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0330 15:10:19.318792 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7bd58c987f-f2glg pod in namespace openshift-ingress (previous: false). I0330 15:10:19.518510 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7bd58c987f-f2glg pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7bd58c987f-f2glg\" is waiting to start: ContainerCreating" I0330 15:10:19.518530 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7bd58c987f-f2glg\" is waiting to start: ContainerCreating" I0330 15:10:19.518540 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-84859f585-7sjf9 pod in namespace openshift-ingress (previous: false). I0330 15:10:19.718809 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-84859f585-7sjf9 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-84859f585-7sjf9\" is waiting to start: ContainerCreating" I0330 15:10:19.718824 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-84859f585-7sjf9\" is waiting to start: ContainerCreating" I0330 15:10:19.718837 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-84859f585-jc2nc pod in namespace openshift-ingress (previous: false). I0330 15:10:19.917525 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-84859f585-jc2nc pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-84859f585-jc2nc\" is waiting to start: ContainerCreating" I0330 15:10:19.917548 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-84859f585-jc2nc\" is waiting to start: ContainerCreating" I0330 15:10:19.917584 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-7ftkc pod in namespace openshift-ingress-canary (previous: false). W0330 15:10:19.925021 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0330 15:10:20.118709 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-7ftkc pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-7ftkc\" is waiting to start: ContainerCreating" I0330 15:10:20.118729 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-7ftkc\" is waiting to start: ContainerCreating" I0330 15:10:20.118759 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-mvfxk pod in namespace openshift-ingress-canary (previous: false). I0330 15:10:20.318306 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-mvfxk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mvfxk\" is waiting to start: ContainerCreating" I0330 15:10:20.318324 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mvfxk\" is waiting to start: ContainerCreating" I0330 15:10:20.318355 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-rn9sx pod in namespace openshift-ingress-canary (previous: false). I0330 15:10:20.517668 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-rn9sx pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rn9sx\" is waiting to start: ContainerCreating" I0330 15:10:20.517685 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rn9sx\" is waiting to start: ContainerCreating" I0330 15:10:20.517703 1 tasks_processing.go:74] worker 50 stopped. I0330 15:10:20.517862 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=31cc0eb0453f2bd1f5d6a338effebd3059adcb4dbaf2cbfdfe75990789de1bc2 I0330 15:10:20.517934 1 recorder.go:75] Recording events/openshift-dns with fingerprint=2d22079e7459f0beffb2de21ac2e8e06f371860a55fc4c52802cefea099d214c I0330 15:10:20.518044 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=c85e2cc6b028f2d978a46e036ad1ff8587f7e907715e4ccfea44de5fa974592a I0330 15:10:20.518082 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=19512e7ace56258b77a145b95f7f848eca0685576de78fdd6b3d4cee8b115ad4 I0330 15:10:20.518138 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=412cbbd8065f0789e3a6572780ceefc4cd700867fa68ad387024355d66e2a3f6 I0330 15:10:20.518165 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=6715eb03aec4f235c94df887115cd9033e2ebe10e138c8373dce68d9c8093bcd I0330 15:10:20.518306 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-8mtnv with fingerprint=3cfcc0cffc6fe22c2b10c3cf367376d02091bbdee0494407637be29c1a8f15b4 I0330 15:10:20.518401 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-sbhtq with fingerprint=84a07f0f8c089c7da7160e9e06cd0b700bb6a787cbd1f03f13caf21f6b7501c8 I0330 15:10:20.518482 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-td6nn with fingerprint=961d1325e1d9907404e74da7ce7ed1dd29bae6e4e5ed0a9f56f275a9921ece3d I0330 15:10:20.518596 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-7879584b45-9gg24 with fingerprint=f0e61cd5e0e903c0935f28ed836b143c6cef7fffc8ec92e71e09c4e18886e200 I0330 15:10:20.518700 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-8f55956f4-9z5w4 with fingerprint=4f5cfd8b113eaa5cd0648b03e9a4668c9bd06e340f1682197ca3aecc19bea33a I0330 15:10:20.518830 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-8f55956f4-rshfr with fingerprint=7b8df60c624d6c55280c7442de9bed1a7299a97b8212d8be0c91c6ae947f1420 I0330 15:10:20.518907 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-7ftkc with fingerprint=549b1aa281d6fa1596273409a2a1ac45ca7394032d8988b5e88bad48637ad018 I0330 15:10:20.518974 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-mvfxk with fingerprint=fccec4665254ad2d274dc494a403d6e3ed66ce6bdaeeed69855648424445ddc8 I0330 15:10:20.519043 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-rn9sx with fingerprint=ed7222a8319d88200c17e5aade60e98d656547dc347fb73cd97a26ca9a35a528 I0330 15:10:20.519058 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.669923768s to process 15 records W0330 15:10:20.925491 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0330 15:10:20.925539 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0330 15:10:20.925561 1 tasks_processing.go:74] worker 1 stopped. E0330 15:10:20.925574 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0330 15:10:20.925589 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0330 15:10:20.925611 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0330 15:10:20.925629 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.075944235s to process 1 records I0330 15:10:28.280762 1 tasks_processing.go:74] worker 46 stopped. I0330 15:10:28.280829 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0330 15:10:28.280844 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.433155681s to process 1 records I0330 15:10:29.053257 1 tasks_processing.go:74] worker 14 stopped. I0330 15:10:29.053514 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=751aa8b7d1208af5fd734152a53c8f2354708612b0cd50cfd9c595651a5cc8c8 I0330 15:10:29.053532 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.206018861s to process 1 records E0330 15:10:29.053586 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.206s with: function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0330 15:10:29.054694 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0330 15:10:29.054707 1 periodic.go:209] Running workloads gatherer I0330 15:10:29.054721 1 tasks_processing.go:45] number of workers: 2 I0330 15:10:29.054726 1 tasks_processing.go:69] worker 1 listening for tasks. I0330 15:10:29.054730 1 tasks_processing.go:71] worker 1 working on workload_info task. I0330 15:10:29.054740 1 tasks_processing.go:69] worker 0 listening for tasks. I0330 15:10:29.054760 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0330 15:10:29.077561 1 tasks_processing.go:74] worker 0 stopped. I0330 15:10:29.077585 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 22.779123ms to process 0 records I0330 15:10:29.077760 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0330 15:10:29.085042 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (9ms) I0330 15:10:29.093132 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (8ms) I0330 15:10:29.100569 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (7ms) I0330 15:10:29.101849 1 configmapobserver.go:84] configmaps "insights-config" not found I0330 15:10:29.108001 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (7ms) I0330 15:10:29.115259 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (7ms) I0330 15:10:29.122718 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (7ms) I0330 15:10:29.129716 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (7ms) I0330 15:10:29.136842 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (7ms) I0330 15:10:29.144009 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (7ms) I0330 15:10:29.151369 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (7ms) I0330 15:10:29.184383 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (33ms) I0330 15:10:29.284845 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0330 15:10:29.385710 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0330 15:10:29.485860 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0330 15:10:29.590117 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (104ms) I0330 15:10:29.685955 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (96ms) I0330 15:10:29.786199 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0330 15:10:29.885286 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (99ms) I0330 15:10:29.985920 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0330 15:10:30.085203 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (99ms) I0330 15:10:30.184705 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (99ms) I0330 15:10:30.285341 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0330 15:10:30.387597 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (102ms) I0330 15:10:30.486109 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (98ms) I0330 15:10:30.585754 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (100ms) I0330 15:10:30.685556 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0330 15:10:30.785369 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0330 15:10:30.885535 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (100ms) I0330 15:10:30.985995 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0330 15:10:31.085813 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0330 15:10:31.185437 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0330 15:10:31.285500 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0330 15:10:31.285531 1 tasks_processing.go:74] worker 1 stopped. E0330 15:10:31.285542 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0330 15:10:31.285894 1 recorder.go:75] Recording config/workload_info with fingerprint=2f4cf0317636f7c2bca18d146a7eb4ff0fded4f4faffacfbdd67470ea03137e8 I0330 15:10:31.285911 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.230793778s to process 1 records E0330 15:10:31.285935 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.231s with: function \"workload_info\" failed with an error" I0330 15:10:31.287036 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0330 15:10:31.287048 1 periodic.go:209] Running conditional gatherer I0330 15:10:31.292254 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0330 15:10:31.298644 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.12:43057->172.30.0.10:53: read: connection refused E0330 15:10:31.298899 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0330 15:10:31.298957 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0330 15:10:31.304040 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0330 15:10:31.304054 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304059 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304063 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304066 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304069 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304072 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304075 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304078 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0330 15:10:31.304081 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0330 15:10:31.304095 1 tasks_processing.go:45] number of workers: 3 I0330 15:10:31.304105 1 tasks_processing.go:69] worker 2 listening for tasks. I0330 15:10:31.304109 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0330 15:10:31.304114 1 tasks_processing.go:69] worker 0 listening for tasks. I0330 15:10:31.304121 1 tasks_processing.go:69] worker 1 listening for tasks. I0330 15:10:31.304135 1 tasks_processing.go:74] worker 1 stopped. I0330 15:10:31.304147 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0330 15:10:31.304149 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0330 15:10:31.304200 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0330 15:10:31.304215 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 817ns to process 1 records I0330 15:10:31.304247 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0330 15:10:31.304255 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.024µs to process 1 records I0330 15:10:31.304260 1 tasks_processing.go:74] worker 0 stopped. I0330 15:10:31.304400 1 tasks_processing.go:74] worker 2 stopped. I0330 15:10:31.304412 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 233.506µs to process 0 records I0330 15:10:31.304433 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.12:43057->172.30.0.10:53: read: connection refused I0330 15:10:31.304452 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0330 15:10:31.324383 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=336eb1c1d8a2588891833439f606434841ae27dc27465f0edb11fb2d86ab791c I0330 15:10:31.324515 1 diskrecorder.go:70] Writing 110 records to /var/lib/insights-operator/insights-2026-03-30-151031.tar.gz I0330 15:10:31.331216 1 diskrecorder.go:51] Wrote 110 records to disk in 6ms I0330 15:10:31.331247 1 periodic.go:278] Gathering cluster info every 2h0m0s I0330 15:10:31.331262 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0330 15:10:32.851302 1 configmapobserver.go:84] configmaps "insights-config" not found I0330 15:10:33.052962 1 configmapobserver.go:84] configmaps "insights-config" not found I0330 15:10:40.825439 1 configmapobserver.go:84] configmaps "insights-config" not found I0330 15:11:20.403420 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="35e15ffe595b482b489377e34c21be57ba2a5ec1cd451da499b4929eb4299ecd") W0330 15:11:20.403457 1 builder.go:160] Restart triggered because of file /var/run/configmaps/service-ca-bundle/service-ca.crt was created I0330 15:11:20.403524 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="46d8cff9be656072bfa3b2b6595edef5ecfadf9ece96edf5245049aad1cddbbd") I0330 15:11:20.403523 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0330 15:11:20.403545 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0330 15:11:20.403576 1 periodic.go:170] Shutting down I0330 15:11:20.403583 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0330 15:11:20.403611 1 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0330 15:11:20.403580 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="49287c7a326df37dfce91ac67ff6720446f42b8fb426ab8b6f7cd71977bff0d2") I0330 15:11:20.403631 1 base_controller.go:181] Shutting down LoggingSyncer ... I0330 15:11:20.403631 1 base_controller.go:181] Shutting down ConfigController ...