W0512 11:37:01.331851 1 cmd.go:257] Using insecure, self-signed certificates I0512 11:37:01.750615 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 11:37:01.750928 1 observer_polling.go:159] Starting file observer I0512 11:37:02.366283 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0512 11:37:02.366477 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0512 11:37:02.367050 1 secure_serving.go:57] Forcing use of http/1.1 only W0512 11:37:02.367069 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0512 11:37:02.367074 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0512 11:37:02.367078 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0512 11:37:02.367081 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0512 11:37:02.367083 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0512 11:37:02.367086 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0512 11:37:02.367075 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0512 11:37:02.370745 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0512 11:37:02.370758 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"68cc66a2-78da-41ad-813c-40d01c64e926", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0512 11:37:02.374560 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0512 11:37:02.374573 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0512 11:37:02.374605 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0512 11:37:02.374621 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 11:37:02.374623 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0512 11:37:02.374633 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 11:37:02.374809 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1029337205/tls.crt::/tmp/serving-cert-1029337205/tls.key" I0512 11:37:02.375165 1 secure_serving.go:213] Serving securely on [::]:8443 I0512 11:37:02.375188 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0512 11:37:02.376409 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0512 11:37:02.376443 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 11:37:02.376540 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0512 11:37:02.387016 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 11:37:02.387035 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 11:37:02.393562 1 secretconfigobserver.go:119] support secret does not exist I0512 11:37:02.398476 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 11:37:02.402980 1 secretconfigobserver.go:119] support secret does not exist I0512 11:37:02.406501 1 recorder.go:161] Pruning old reports every 7h39m6s, max age is 288h0m0s I0512 11:37:02.411446 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0512 11:37:02.411462 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0512 11:37:02.411475 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0512 11:37:02.411485 1 periodic.go:209] Running clusterconfig gatherer I0512 11:37:02.411488 1 insightsreport.go:296] Starting report retriever I0512 11:37:02.411496 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0512 11:37:02.411544 1 tasks_processing.go:45] number of workers: 64 I0512 11:37:02.411562 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 11:37:02.411573 1 tasks_processing.go:69] worker 8 listening for tasks. I0512 11:37:02.411576 1 tasks_processing.go:69] worker 4 listening for tasks. I0512 11:37:02.411584 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 11:37:02.411584 1 tasks_processing.go:69] worker 16 listening for tasks. I0512 11:37:02.411594 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 11:37:02.411597 1 tasks_processing.go:69] worker 13 listening for tasks. I0512 11:37:02.411604 1 tasks_processing.go:69] worker 14 listening for tasks. I0512 11:37:02.411606 1 tasks_processing.go:69] worker 15 listening for tasks. I0512 11:37:02.411613 1 tasks_processing.go:71] worker 14 working on version task. I0512 11:37:02.411656 1 tasks_processing.go:69] worker 5 listening for tasks. I0512 11:37:02.411662 1 tasks_processing.go:71] worker 5 working on nodes task. I0512 11:37:02.411663 1 tasks_processing.go:69] worker 11 listening for tasks. I0512 11:37:02.411689 1 tasks_processing.go:69] worker 32 listening for tasks. I0512 11:37:02.411683 1 tasks_processing.go:69] worker 7 listening for tasks. I0512 11:37:02.411698 1 tasks_processing.go:69] worker 33 listening for tasks. I0512 11:37:02.411689 1 tasks_processing.go:69] worker 6 listening for tasks. I0512 11:37:02.411700 1 tasks_processing.go:69] worker 29 listening for tasks. I0512 11:37:02.411705 1 tasks_processing.go:69] worker 35 listening for tasks. I0512 11:37:02.411676 1 tasks_processing.go:69] worker 34 listening for tasks. I0512 11:37:02.411709 1 tasks_processing.go:69] worker 63 listening for tasks. I0512 11:37:02.411705 1 tasks_processing.go:69] worker 30 listening for tasks. I0512 11:37:02.411710 1 tasks_processing.go:69] worker 37 listening for tasks. I0512 11:37:02.411720 1 tasks_processing.go:71] worker 0 working on oauths task. I0512 11:37:02.411724 1 tasks_processing.go:69] worker 51 listening for tasks. I0512 11:37:02.411730 1 tasks_processing.go:69] worker 36 listening for tasks. I0512 11:37:02.411732 1 tasks_processing.go:69] worker 39 listening for tasks. I0512 11:37:02.411737 1 tasks_processing.go:69] worker 48 listening for tasks. I0512 11:37:02.411743 1 tasks_processing.go:71] worker 2 working on certificate_signing_requests task. I0512 11:37:02.411746 1 tasks_processing.go:71] worker 16 working on tsdb_status task. I0512 11:37:02.411749 1 tasks_processing.go:69] worker 50 listening for tasks. W0512 11:37:02.411773 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 11:37:02.411791 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 33.244µs to process 0 records I0512 11:37:02.411798 1 tasks_processing.go:71] worker 13 working on openstack_version task. I0512 11:37:02.411804 1 tasks_processing.go:69] worker 28 listening for tasks. I0512 11:37:02.411813 1 tasks_processing.go:69] worker 20 listening for tasks. I0512 11:37:02.411816 1 tasks_processing.go:69] worker 23 listening for tasks. I0512 11:37:02.411819 1 tasks_processing.go:69] worker 19 listening for tasks. I0512 11:37:02.411824 1 tasks_processing.go:69] worker 24 listening for tasks. I0512 11:37:02.411825 1 tasks_processing.go:69] worker 60 listening for tasks. I0512 11:37:02.411831 1 tasks_processing.go:69] worker 59 listening for tasks. I0512 11:37:02.411709 1 tasks_processing.go:69] worker 40 listening for tasks. I0512 11:37:02.411842 1 tasks_processing.go:69] worker 18 listening for tasks. I0512 11:37:02.411714 1 tasks_processing.go:71] worker 8 working on feature_gates task. I0512 11:37:02.411837 1 tasks_processing.go:69] worker 49 listening for tasks. I0512 11:37:02.411850 1 tasks_processing.go:69] worker 58 listening for tasks. I0512 11:37:02.411855 1 tasks_processing.go:69] worker 21 listening for tasks. I0512 11:37:02.411720 1 tasks_processing.go:69] worker 22 listening for tasks. I0512 11:37:02.411724 1 tasks_processing.go:69] worker 38 listening for tasks. I0512 11:37:02.411864 1 tasks_processing.go:69] worker 27 listening for tasks. I0512 11:37:02.411866 1 tasks_processing.go:69] worker 62 listening for tasks. I0512 11:37:02.411723 1 tasks_processing.go:69] worker 56 listening for tasks. I0512 11:37:02.411872 1 tasks_processing.go:69] worker 43 listening for tasks. I0512 11:37:02.411730 1 tasks_processing.go:69] worker 57 listening for tasks. I0512 11:37:02.411874 1 tasks_processing.go:69] worker 52 listening for tasks. I0512 11:37:02.411875 1 tasks_processing.go:69] worker 54 listening for tasks. I0512 11:37:02.411879 1 tasks_processing.go:69] worker 41 listening for tasks. I0512 11:37:02.411708 1 tasks_processing.go:71] worker 1 working on mutating_webhook_configurations task. I0512 11:37:02.411886 1 tasks_processing.go:69] worker 45 listening for tasks. I0512 11:37:02.411887 1 tasks_processing.go:69] worker 46 listening for tasks. I0512 11:37:02.411890 1 tasks_processing.go:69] worker 44 listening for tasks. I0512 11:37:02.411692 1 tasks_processing.go:69] worker 31 listening for tasks. I0512 11:37:02.411862 1 tasks_processing.go:69] worker 61 listening for tasks. I0512 11:37:02.411898 1 tasks_processing.go:69] worker 9 listening for tasks. I0512 11:37:02.411867 1 tasks_processing.go:69] worker 53 listening for tasks. I0512 11:37:02.411840 1 tasks_processing.go:69] worker 25 listening for tasks. I0512 11:37:02.411880 1 tasks_processing.go:69] worker 42 listening for tasks. I0512 11:37:02.411714 1 tasks_processing.go:71] worker 4 working on machine_sets task. I0512 11:37:02.411919 1 tasks_processing.go:69] worker 47 listening for tasks. I0512 11:37:02.411925 1 tasks_processing.go:69] worker 55 listening for tasks. I0512 11:37:02.411932 1 tasks_processing.go:69] worker 10 listening for tasks. I0512 11:37:02.411614 1 tasks_processing.go:71] worker 15 working on service_accounts task. I0512 11:37:02.411944 1 tasks_processing.go:69] worker 3 listening for tasks. I0512 11:37:02.411963 1 tasks_processing.go:69] worker 12 listening for tasks. I0512 11:37:02.411974 1 tasks_processing.go:71] worker 48 working on storage_cluster task. I0512 11:37:02.411857 1 tasks_processing.go:69] worker 26 listening for tasks. I0512 11:37:02.411979 1 tasks_processing.go:71] worker 12 working on qemu_kubevirt_launcher_logs task. I0512 11:37:02.411985 1 tasks_processing.go:71] worker 28 working on nodenetworkstates task. I0512 11:37:02.412003 1 tasks_processing.go:71] worker 26 working on machines task. I0512 11:37:02.412010 1 tasks_processing.go:71] worker 20 working on monitoring_persistent_volumes task. I0512 11:37:02.412022 1 tasks_processing.go:71] worker 35 working on openstack_dataplanedeployments task. I0512 11:37:02.412039 1 tasks_processing.go:71] worker 31 working on support_secret task. I0512 11:37:02.412044 1 tasks_processing.go:71] worker 25 working on jaegers task. I0512 11:37:02.412149 1 tasks_processing.go:71] worker 7 working on cost_management_metrics_configs task. I0512 11:37:02.412158 1 tasks_processing.go:71] worker 11 working on sap_pods task. I0512 11:37:02.412185 1 tasks_processing.go:71] worker 42 working on openshift_logging task. I0512 11:37:02.411727 1 tasks_processing.go:69] worker 17 listening for tasks. I0512 11:37:02.412196 1 tasks_processing.go:71] worker 17 working on crds task. I0512 11:37:02.411983 1 tasks_processing.go:71] worker 16 working on image_registries task. I0512 11:37:02.412345 1 tasks_processing.go:71] worker 41 working on authentication task. I0512 11:37:02.412352 1 tasks_processing.go:71] worker 55 working on storage_classes task. I0512 11:37:02.412357 1 tasks_processing.go:71] worker 32 working on silenced_alerts task. W0512 11:37:02.412384 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 11:37:02.412406 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 34.789µs to process 0 records I0512 11:37:02.412420 1 tasks_processing.go:71] worker 46 working on openstack_dataplanenodesets task. I0512 11:37:02.412555 1 tasks_processing.go:71] worker 9 working on olm_operators task. I0512 11:37:02.412628 1 tasks_processing.go:71] worker 49 working on clusterroles task. I0512 11:37:02.411980 1 tasks_processing.go:71] worker 50 working on machine_healthchecks task. I0512 11:37:02.412786 1 tasks_processing.go:71] worker 6 working on proxies task. I0512 11:37:02.412838 1 tasks_processing.go:71] worker 34 working on ingress task. I0512 11:37:02.412881 1 tasks_processing.go:74] worker 32 stopped. I0512 11:37:02.412895 1 tasks_processing.go:71] worker 52 working on machine_config_pools task. I0512 11:37:02.412995 1 tasks_processing.go:71] worker 63 working on infrastructures task. I0512 11:37:02.412004 1 tasks_processing.go:71] worker 43 working on metrics task. W0512 11:37:02.413197 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 11:37:02.413206 1 tasks_processing.go:74] worker 43 stopped. I0512 11:37:02.413214 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 20.37µs to process 0 records I0512 11:37:02.413220 1 tasks_processing.go:71] worker 44 working on image task. I0512 11:37:02.413271 1 tasks_processing.go:71] worker 53 working on ceph_cluster task. I0512 11:37:02.413308 1 tasks_processing.go:71] worker 61 working on node_logs task. I0512 11:37:02.413331 1 tasks_processing.go:71] worker 30 working on ingress_certificates task. I0512 11:37:02.413342 1 tasks_processing.go:71] worker 10 working on operators task. I0512 11:37:02.413347 1 tasks_processing.go:71] worker 33 working on pdbs task. I0512 11:37:02.413364 1 tasks_processing.go:71] worker 29 working on pod_network_connectivity_checks task. I0512 11:37:02.413371 1 tasks_processing.go:71] worker 60 working on container_images task. I0512 11:37:02.413465 1 tasks_processing.go:71] worker 45 working on machine_configs task. I0512 11:37:02.413497 1 tasks_processing.go:71] worker 40 working on active_alerts task. I0512 11:37:02.413527 1 tasks_processing.go:71] worker 36 working on lokistack task. W0512 11:37:02.413543 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 11:37:02.413559 1 tasks_processing.go:74] worker 40 stopped. I0512 11:37:02.413568 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 49.104µs to process 0 records I0512 11:37:02.413576 1 tasks_processing.go:71] worker 59 working on machine_autoscalers task. I0512 11:37:02.413662 1 tasks_processing.go:71] worker 58 working on validating_webhook_configurations task. I0512 11:37:02.413669 1 tasks_processing.go:71] worker 39 working on operators_pods_and_events task. I0512 11:37:02.411976 1 tasks_processing.go:71] worker 3 working on aggregated_monitoring_cr_names task. I0512 11:37:02.413702 1 tasks_processing.go:71] worker 37 working on sap_config task. I0512 11:37:02.413713 1 tasks_processing.go:71] worker 18 working on container_runtime_configs task. I0512 11:37:02.413743 1 tasks_processing.go:71] worker 54 working on install_plans task. I0512 11:37:02.413802 1 tasks_processing.go:71] worker 23 working on dvo_metrics task. I0512 11:37:02.413808 1 tasks_processing.go:71] worker 51 working on openstack_controlplanes task. I0512 11:37:02.413852 1 tasks_processing.go:71] worker 21 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0512 11:37:02.414425 1 tasks_processing.go:71] worker 57 working on networks task. I0512 11:37:02.414492 1 tasks_processing.go:71] worker 19 working on overlapping_namespace_uids task. I0512 11:37:02.414673 1 tasks_processing.go:71] worker 22 working on schedulers task. I0512 11:37:02.414801 1 tasks_processing.go:71] worker 24 working on cluster_apiserver task. I0512 11:37:02.414940 1 tasks_processing.go:71] worker 56 working on image_pruners task. I0512 11:37:02.412348 1 tasks_processing.go:71] worker 47 working on config_maps task. I0512 11:37:02.414988 1 tasks_processing.go:71] worker 38 working on sap_datahubs task. I0512 11:37:02.413858 1 tasks_processing.go:71] worker 27 working on nodenetworkconfigurationpolicies task. I0512 11:37:02.414707 1 tasks_processing.go:71] worker 62 working on openshift_machine_api_events task. I0512 11:37:02.416201 1 tasks_processing.go:74] worker 13 stopped. I0512 11:37:02.416218 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 4.391762ms to process 0 records I0512 11:37:02.416274 1 tasks_processing.go:74] worker 4 stopped. I0512 11:37:02.416284 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 4.354572ms to process 0 records I0512 11:37:02.417840 1 tasks_processing.go:74] worker 28 stopped. I0512 11:37:02.417855 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 5.841157ms to process 0 records I0512 11:37:02.419034 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0512 11:37:02.419050 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 11:37:02.419055 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0512 11:37:02.419058 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0512 11:37:02.419071 1 controller.go:489] The operator is still being initialized I0512 11:37:02.419076 1 controller.go:512] The operator is healthy I0512 11:37:02.427300 1 tasks_processing.go:74] worker 8 stopped. I0512 11:37:02.427431 1 recorder.go:75] Recording config/featuregate with fingerprint=3c161eeb01906d8cfb335af0e3317e7f169b96c3919641ae1a8e815de8acede3 I0512 11:37:02.427443 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 15.441546ms to process 1 records I0512 11:37:02.444371 1 tasks_processing.go:74] worker 7 stopped. I0512 11:37:02.444385 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 32.203525ms to process 0 records I0512 11:37:02.444392 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 32.180512ms to process 0 records I0512 11:37:02.444398 1 tasks_processing.go:74] worker 42 stopped. I0512 11:37:02.444432 1 tasks_processing.go:74] worker 35 stopped. I0512 11:37:02.444449 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 32.394853ms to process 0 records E0512 11:37:02.444465 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0512 11:37:02.444476 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 31.066471ms to process 0 records I0512 11:37:02.444478 1 tasks_processing.go:74] worker 29 stopped. I0512 11:37:02.444552 1 tasks_processing.go:74] worker 50 stopped. E0512 11:37:02.444561 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0512 11:37:02.444568 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 31.84984ms to process 0 records I0512 11:37:02.444687 1 tasks_processing.go:74] worker 0 stopped. I0512 11:37:02.444916 1 recorder.go:75] Recording config/oauth with fingerprint=164e438e43e84b8ff2532f290465e0442a054208dae2606178c885e941ec8dfb I0512 11:37:02.444932 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 32.950967ms to process 1 records I0512 11:37:02.444941 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 31.384788ms to process 0 records I0512 11:37:02.444947 1 tasks_processing.go:74] worker 36 stopped. I0512 11:37:02.445027 1 tasks_processing.go:74] worker 25 stopped. I0512 11:37:02.445036 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 32.974733ms to process 0 records I0512 11:37:02.445043 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 32.862565ms to process 0 records E0512 11:37:02.445048 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0512 11:37:02.445052 1 gather.go:177] gatherer "clusterconfig" function "machines" took 33.02125ms to process 0 records I0512 11:37:02.445058 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 33.052299ms to process 0 records I0512 11:37:02.445065 1 tasks_processing.go:74] worker 48 stopped. I0512 11:37:02.445070 1 tasks_processing.go:74] worker 11 stopped. I0512 11:37:02.445074 1 tasks_processing.go:74] worker 26 stopped. I0512 11:37:02.445117 1 tasks_processing.go:74] worker 51 stopped. I0512 11:37:02.445130 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 31.298703ms to process 0 records I0512 11:37:02.445138 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 31.409522ms to process 0 records I0512 11:37:02.445151 1 tasks_processing.go:74] worker 37 stopped. I0512 11:37:02.445209 1 tasks_processing.go:74] worker 6 stopped. I0512 11:37:02.445284 1 recorder.go:75] Recording config/proxy with fingerprint=988a801904f6598a57a07f32a096f6f06a697ec942f29541eb62bc3c83ee7dc8 I0512 11:37:02.445299 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 32.411214ms to process 1 records I0512 11:37:02.445311 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 31.533428ms to process 0 records I0512 11:37:02.445318 1 tasks_processing.go:74] worker 18 stopped. I0512 11:37:02.445584 1 tasks_processing.go:74] worker 41 stopped. I0512 11:37:02.445795 1 recorder.go:75] Recording config/authentication with fingerprint=a1b07b7d2ce84bf7a7dae5d140df46592b53d2effc708f0222367482c20385a0 I0512 11:37:02.445811 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 33.221634ms to process 1 records I0512 11:37:02.449898 1 tasks_processing.go:74] worker 27 stopped. I0512 11:37:02.449907 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 34.620272ms to process 0 records I0512 11:37:02.450459 1 tasks_processing.go:74] worker 20 stopped. I0512 11:37:02.450472 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 38.441617ms to process 0 records I0512 11:37:02.450506 1 tasks_processing.go:74] worker 31 stopped. E0512 11:37:02.450535 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0512 11:37:02.450542 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 38.45337ms to process 0 records I0512 11:37:02.450646 1 tasks_processing.go:74] worker 61 stopped. I0512 11:37:02.450658 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 37.322226ms to process 0 records I0512 11:37:02.450950 1 tasks_processing.go:74] worker 56 stopped. I0512 11:37:02.451188 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=d0a38ac0349d51461294c3a674b8fa2333d50bdaf32a11125ddc962f2faae6b5 I0512 11:37:02.451203 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 35.970765ms to process 1 records I0512 11:37:02.451756 1 tasks_processing.go:74] worker 59 stopped. I0512 11:37:02.451771 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 38.168976ms to process 0 records I0512 11:37:02.451922 1 tasks_processing.go:74] worker 9 stopped. I0512 11:37:02.451935 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 39.350905ms to process 0 records I0512 11:37:02.451945 1 tasks_processing.go:74] worker 34 stopped. I0512 11:37:02.452050 1 recorder.go:75] Recording config/ingress with fingerprint=2537d4b9787fc8f391d8cb019c2f05644f24cea4cc3f887713cd747af437f903 I0512 11:37:02.452063 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 39.092562ms to process 1 records I0512 11:37:02.452157 1 recorder.go:75] Recording config/apiserver with fingerprint=def5060563dc17685cb4adf2049d512018dfed603e2b76ab892faaedc7e68cbc I0512 11:37:02.452165 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 37.111827ms to process 1 records I0512 11:37:02.452171 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 36.804859ms to process 0 records I0512 11:37:02.452176 1 tasks_processing.go:74] worker 38 stopped. I0512 11:37:02.452178 1 tasks_processing.go:74] worker 24 stopped. I0512 11:37:02.452194 1 tasks_processing.go:74] worker 33 stopped. I0512 11:37:02.452298 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=40d5d3ac915dc4c8540fb5ea514e40df20d03f7d1bd6a6b378ac2a8a3409a94e I0512 11:37:02.452323 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=f712cd194044f9277ffc39480587609b4547510a5b88bed54703e82d1aa234dd I0512 11:37:02.452342 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=69f73aef3c52ec97e5f4dc1a64ce025a3433fa78b8049b853244e6c82f1a9496 I0512 11:37:02.452352 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 38.836084ms to process 3 records I0512 11:37:02.452470 1 tasks_processing.go:74] worker 63 stopped. I0512 11:37:02.452614 1 gather_logs.go:145] no pods in namespace were found I0512 11:37:02.453007 1 recorder.go:75] Recording config/infrastructure with fingerprint=8075efc29e072b50b5ec5b8cdde5f44fec0a3fa66e4b0dace93291f794613488 I0512 11:37:02.453021 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 39.20321ms to process 1 records I0512 11:37:02.453033 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 40.064585ms to process 0 records I0512 11:37:02.453038 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 39.224351ms to process 0 records I0512 11:37:02.453064 1 tasks_processing.go:74] worker 53 stopped. I0512 11:37:02.453051 1 tasks_processing.go:74] worker 46 stopped. I0512 11:37:02.453108 1 recorder.go:75] Recording config/image with fingerprint=c23baefaaa0b4033721b0e846c92190426a22051cc5d2f937cfb4a123cd21589 I0512 11:37:02.453118 1 gather.go:177] gatherer "clusterconfig" function "image" took 39.317757ms to process 1 records I0512 11:37:02.453124 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 40.636824ms to process 0 records I0512 11:37:02.453124 1 tasks_processing.go:74] worker 44 stopped. I0512 11:37:02.453133 1 tasks_processing.go:74] worker 12 stopped. I0512 11:37:02.453198 1 tasks_processing.go:74] worker 22 stopped. I0512 11:37:02.453214 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=a14e0cf6b7323fb0543ac88c9b1eba4d56700e34f8dd1d591a24548ed5795953 I0512 11:37:02.453222 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 38.026314ms to process 1 records I0512 11:37:02.453310 1 tasks_processing.go:74] worker 57 stopped. I0512 11:37:02.453620 1 recorder.go:75] Recording config/network with fingerprint=dbe3017705e2e4956bba3a88f71b699ef4e87703f8b8115b8403637b6800a13d I0512 11:37:02.453720 1 gather.go:177] gatherer "clusterconfig" function "networks" took 38.350571ms to process 1 records I0512 11:37:02.453851 1 tasks_processing.go:74] worker 16 stopped. I0512 11:37:02.454528 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=94f003e42a84cdb0af0ae9334a3887ff7d71cce87309372404d8bd85f22e602e I0512 11:37:02.454610 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 40.78297ms to process 1 records I0512 11:37:02.454649 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 39.285081ms to process 0 records I0512 11:37:02.454669 1 tasks_processing.go:74] worker 62 stopped. I0512 11:37:02.454819 1 tasks_processing.go:74] worker 55 stopped. I0512 11:37:02.455030 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=526167b12cb9b2fa1b0442c557e34e05e74f8232a4faa44ed28a1f2bcc392617 I0512 11:37:02.455064 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=48763a34085f007f2e9a1c47b9e2b6da987760ff018000c171a501048c5d1377 I0512 11:37:02.455082 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 42.4557ms to process 2 records I0512 11:37:02.455904 1 tasks_processing.go:74] worker 2 stopped. I0512 11:37:02.455916 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 44.151856ms to process 0 records I0512 11:37:02.455996 1 tasks_processing.go:74] worker 1 stopped. I0512 11:37:02.456092 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=3fd83a269518d1303d92c3984b610e86fb9e6fafb4d5914341e667c100d54a79 I0512 11:37:02.456130 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=19ebe3f1c0cb5cb94a20a7170b07ce7849d91eda7e329f1230cfadb2cc3d41ea I0512 11:37:02.456161 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=3111fcfba2b57bb0f2f1f3b10e0cfeb6e7e9b5ae68af1a29eb0dbe16edc71d59 I0512 11:37:02.456169 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 44.021685ms to process 3 records I0512 11:37:02.457801 1 tasks_processing.go:74] worker 58 stopped. I0512 11:37:02.457955 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=c3dae84bc8304337d5ae9e20c829043ff120664b72e6b1944cc6f661c606caa3 I0512 11:37:02.458046 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=95e2776938d6b3f121331751e7a139672c3d39b61f6d5ac13f6eaa082bbb563c I0512 11:37:02.458071 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=4e2d96ca511bf299e190fd8eb4a36a01fd7f068ee8ef43d39160eed97ae88a27 I0512 11:37:02.458108 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=f20b8714ae05c2dcc6573f15cdb9412ee6df8c7022dd8c5e8d1f2b181ee9892f I0512 11:37:02.458139 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=fac4174da3764b23d5ea335bb695a6439f7468560b3766987985daef3f0ef2f7 I0512 11:37:02.458179 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=8eda1b642cabae776e1e5e78050d5e9a81c08394d90ccc654bcb491cdfd42a04 I0512 11:37:02.458212 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=dcb2cdaaa8ef95ba2a67692ae0352f1d3f1d0d8c3e0625226e8a31a17ecf9b0a I0512 11:37:02.458260 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=61542b5d7f53e29598b8ef53038d2ee9ffd52009c7b7d5a14ca2c619425d09a8 I0512 11:37:02.458298 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=83938ad2e9fc00ffd1e6cfbad6b46823b9e49eebe0374a8961d1f2a8b0d675ac I0512 11:37:02.458337 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=d9b05b499e723ab7d8b5c606b2e9344303c438b2f2efbff16bda4656bf661425 I0512 11:37:02.458366 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=866158a0a8ff4099a83eeb1f2a08435bb9dcfd5a5c5db2da2a77c59c6a7f452d I0512 11:37:02.458374 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 44.118519ms to process 11 records I0512 11:37:02.458455 1 tasks_processing.go:74] worker 5 stopped. I0512 11:37:02.458744 1 recorder.go:75] Recording config/node/ip-10-0-0-96.ec2.internal with fingerprint=91bae6141c47dbba8d3ad7cbf1e5bec586b98a83970faba3c4ef353bdea1dd29 I0512 11:37:02.458808 1 recorder.go:75] Recording config/node/ip-10-0-1-229.ec2.internal with fingerprint=db59a041978e7ff2c4b766161d623ab7002da5a2ff55620b6c8bd57574f4754c I0512 11:37:02.458879 1 recorder.go:75] Recording config/node/ip-10-0-2-249.ec2.internal with fingerprint=5d505a61ad95e45341a0a93485e9ffa29f514b797a043fa9494f358efe925689 I0512 11:37:02.458887 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 46.222421ms to process 3 records I0512 11:37:02.458904 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0512 11:37:02.458912 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 43.745289ms to process 1 records I0512 11:37:02.458915 1 tasks_processing.go:74] worker 19 stopped. W0512 11:37:02.459787 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 11:37:02.469467 1 tasks_processing.go:74] worker 52 stopped. I0512 11:37:02.469489 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 56.55492ms to process 0 records I0512 11:37:02.469604 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0512 11:37:02.469610 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0512 11:37:02.469724 1 tasks_processing.go:74] worker 49 stopped. W0512 11:37:02.469793 1 operator.go:288] started I0512 11:37:02.469817 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0512 11:37:02.469900 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=37b5ac4f3f3cdf7d55f1a69b66b4c61542fefc1fa2eb7c2fc47aea41c5b42ee3 I0512 11:37:02.470010 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=9f403f2cbf8f5ddd0f033947c649294a7364c2049dfd0e0ba6634ba762980eb7 I0512 11:37:02.470019 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 57.083848ms to process 2 records I0512 11:37:02.473999 1 tasks_processing.go:74] worker 17 stopped. I0512 11:37:02.474509 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=cd54829a63f382192744ff9dd456a2b7abe0daf6625f68503b45e8d857ac2c49 I0512 11:37:02.474754 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=2dbad195e5ae07aba285c7905dc869690e51c59a68d3034aa627efa57478cba7 I0512 11:37:02.474765 1 gather.go:177] gatherer "clusterconfig" function "crds" took 61.786452ms to process 2 records I0512 11:37:02.474788 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 11:37:02.474864 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 11:37:02.474885 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0512 11:37:02.476123 1 tasks_processing.go:74] worker 60 stopped. I0512 11:37:02.476256 1 recorder.go:75] Recording config/running_containers with fingerprint=02cd720345f1c731d648448db465d3047df2f6798fe295a736b24ccf59a21f90 I0512 11:37:02.476283 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 62.728022ms to process 1 records I0512 11:37:02.476975 1 base_controller.go:82] Caches are synced for ConfigController I0512 11:37:02.476986 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0512 11:37:02.478384 1 tasks_processing.go:74] worker 3 stopped. I0512 11:37:02.478401 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 64.685773ms to process 0 records I0512 11:37:02.478451 1 tasks_processing.go:74] worker 21 stopped. I0512 11:37:02.478465 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 64.035448ms to process 0 records I0512 11:37:02.478840 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0512 11:37:02.478852 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 11:37:02.478857 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0512 11:37:02.478859 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0512 11:37:02.478863 1 controller.go:212] Source scaController *sca.Controller is not ready I0512 11:37:02.478880 1 controller.go:489] The operator is still being initialized I0512 11:37:02.478887 1 controller.go:512] The operator is healthy I0512 11:37:02.483243 1 prometheus_rules.go:88] Prometheus rules successfully created I0512 11:37:02.483882 1 configmapobserver.go:84] configmaps "insights-config" not found E0512 11:37:02.487494 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%274fea5d7d-e8a3-402b-b7e9-474eee240c3c%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:43817->172.30.0.10:53: read: connection refused I0512 11:37:02.487507 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%274fea5d7d-e8a3-402b-b7e9-474eee240c3c%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:43817->172.30.0.10:53: read: connection refused I0512 11:37:02.494861 1 tasks_processing.go:74] worker 47 stopped. E0512 11:37:02.494873 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0512 11:37:02.494883 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0512 11:37:02.494887 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0512 11:37:02.494899 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0512 11:37:02.494931 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0512 11:37:02.494938 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0512 11:37:02.494943 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0512 11:37:02.494947 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0512 11:37:02.494984 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0512 11:37:02.494992 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0512 11:37:02.494997 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 79.853839ms to process 7 records I0512 11:37:02.527022 1 tasks_processing.go:74] worker 14 stopped. I0512 11:37:02.527340 1 recorder.go:75] Recording config/version with fingerprint=a95ef236fe3766de6531d2d81528a169c509d93c4a6cc0a7e43fdfa7288c2468 I0512 11:37:02.527358 1 recorder.go:75] Recording config/id with fingerprint=5f47ef76537256d4fe3637d7209acd694433dc3fd6d5870917d447c3eb330693 I0512 11:37:02.527365 1 gather.go:177] gatherer "clusterconfig" function "version" took 115.394367ms to process 2 records I0512 11:37:02.544362 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0512 11:37:02.547493 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:53500->172.30.0.10:53: read: connection refused I0512 11:37:02.547506 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:53500->172.30.0.10:53: read: connection refused I0512 11:37:02.569849 1 base_controller.go:82] Caches are synced for LoggingSyncer I0512 11:37:02.569860 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0512 11:37:02.577637 1 tasks_processing.go:74] worker 30 stopped. E0512 11:37:02.577657 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0512 11:37:02.577663 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q8btjun6td6kc8ulg3ih7ojvp65gmqk-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q8btjun6td6kc8ulg3ih7ojvp65gmqk-primary-cert-bundle-secret" not found I0512 11:37:02.577726 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=32085e338468e4f60a89be91f0e53d456a709a17317aa7f29a5872a82568ef6d I0512 11:37:02.577739 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 164.292389ms to process 1 records W0512 11:37:03.458956 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 11:37:03.487565 1 tasks_processing.go:74] worker 45 stopped. I0512 11:37:03.487606 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0512 11:37:03.487619 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 1.074079382s to process 1 records I0512 11:37:03.666874 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0512 11:37:03.666892 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0512 11:37:03.667593 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-29tvk pod in namespace openshift-dns (previous: false). I0512 11:37:03.890187 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0512 11:37:03.901741 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-29tvk pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-29tvk\" is waiting to start: ContainerCreating" I0512 11:37:03.901758 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-29tvk\" is waiting to start: ContainerCreating" I0512 11:37:03.901766 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-29tvk pod in namespace openshift-dns (previous: false). I0512 11:37:04.073452 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-29tvk pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-29tvk\" is waiting to start: ContainerCreating" I0512 11:37:04.073470 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-29tvk\" is waiting to start: ContainerCreating" I0512 11:37:04.073482 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-4h6xl pod in namespace openshift-dns (previous: false). I0512 11:37:04.318624 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-4h6xl pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-4h6xl\" is waiting to start: ContainerCreating" I0512 11:37:04.318645 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-4h6xl\" is waiting to start: ContainerCreating" I0512 11:37:04.318654 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-4h6xl pod in namespace openshift-dns (previous: false). W0512 11:37:04.458064 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 11:37:04.476108 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-4h6xl pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-4h6xl\" is waiting to start: ContainerCreating" I0512 11:37:04.476124 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-4h6xl\" is waiting to start: ContainerCreating" I0512 11:37:04.476139 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-xm89w pod in namespace openshift-dns (previous: false). I0512 11:37:04.493137 1 tasks_processing.go:74] worker 10 stopped. I0512 11:37:04.493187 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=7c4ef6dc85567f56f0f8d0db326ee1cbf818244bd8314ebc5fac0aa44116eaac I0512 11:37:04.493217 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=909e6f42142e3b7f39505036f91b0bf6bd4ee60449bd014b23ae95f0ce694f93 I0512 11:37:04.493247 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0512 11:37:04.493276 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=05094153febfdfd652818ed71921edd1d9fba6408ee34b7ea69603e92031f978 I0512 11:37:04.493292 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0512 11:37:04.493313 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=2b7af85c0d9198a21ff002abb316d0ac80548d4dcef7f4569526977a84840e65 I0512 11:37:04.493359 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=22691a34475762c1358761179e3aa00bda0492e52016c8931056b37c73b5c831 I0512 11:37:04.493382 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=8a0434674caaf6041967bc1fee785988415e5440694ff1a58e5513c26fb5faee I0512 11:37:04.493398 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=c923a9c04622e7bf337799a638f6f9b7571d8d842120f8148cd907f6bf2aa48c I0512 11:37:04.493417 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=acf1064d07d8909952e5d24560a95a20ba0b6657c3893472fb3bfc952dd24bb1 I0512 11:37:04.493426 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0512 11:37:04.493442 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=5fb15133d8b896fc8955b9ac5b62ed2851d1a14f1ed8e2311a36075b8c8da6dd I0512 11:37:04.493451 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0512 11:37:04.493469 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=090bedf74cb7629e036e90e93850adb1be3e180bff31789a61990f6c096eec48 I0512 11:37:04.493478 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0512 11:37:04.493497 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=7f331528452e517269a64a29ef1a33d07426b9b5e064b1c2582de28d03b46a17 I0512 11:37:04.493507 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0512 11:37:04.493537 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=3ed08b3973bf3024ae265c89aeaabcf579b67b6cfc868be2abe0bdb477674b39 I0512 11:37:04.493665 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=40f5bce1655902e152fd0c7e8b8f1abd3b1203488722a54b439f2b22718c1eed I0512 11:37:04.493675 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0512 11:37:04.493683 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0512 11:37:04.493712 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0512 11:37:04.493736 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=d02ce3b937bddc8eec10fb187b67447f9d25724f6fc54ad69613d76291350fa6 I0512 11:37:04.493760 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=014959fefbccefcc490c43a46582309dc21c91ae3c40d59a50e03f110766cb8c I0512 11:37:04.493769 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0512 11:37:04.493785 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=3084a243df0475bed2e7bed2d2828692786759f8c30aeb7ebf3f5ebe5bb35cb4 I0512 11:37:04.493796 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0512 11:37:04.493809 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=9c47ee90a746f1eccbe910fb58da894bb70bc3615b7245a53280fe1ea602d52b I0512 11:37:04.493824 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=9c10d0cdcfad5d6809877d78c667307e5636a6effea6867f69a138f26a74ef62 I0512 11:37:04.493837 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=833419c3e56cadb6cdf85c4ba390426d0f8c1e197ad88ce31c0c6ba29f49e8ac I0512 11:37:04.493853 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=b1b0a44c993fe91fd33c0e02e6b3ee3457ae7a5581af001bcb75f428c97b15d3 I0512 11:37:04.493875 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=90e66df83f1ae76863ac074a71194e80b2e4543789b4489e8b5b55850dd8799d I0512 11:37:04.493883 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0512 11:37:04.493906 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=d0fb711cf147a8a2a8390271ccf8bd88e5f5b620caa94eb0c499481cacd44637 I0512 11:37:04.493922 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0512 11:37:04.493930 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0512 11:37:04.493937 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.079777661s to process 36 records I0512 11:37:04.693903 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xm89w pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-xm89w\" is waiting to start: ContainerCreating" I0512 11:37:04.693921 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-xm89w\" is waiting to start: ContainerCreating" I0512 11:37:04.693934 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-xm89w pod in namespace openshift-dns (previous: false). I0512 11:37:04.872168 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xm89w pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-xm89w\" is waiting to start: ContainerCreating" I0512 11:37:04.872185 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-xm89w\" is waiting to start: ContainerCreating" I0512 11:37:04.872195 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-6vswq pod in namespace openshift-dns (previous: false). I0512 11:37:05.073086 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 11:37:05.073105 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-d2t78 pod in namespace openshift-dns (previous: false). I0512 11:37:05.274032 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 11:37:05.274053 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-kr9pq pod in namespace openshift-dns (previous: false). W0512 11:37:05.458992 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 11:37:05.473264 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 11:37:05.473316 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5965dcf9f8-kb75q pod in namespace openshift-image-registry (previous: false). I0512 11:37:05.673978 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5965dcf9f8-kb75q pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5965dcf9f8-kb75q\" is waiting to start: ContainerCreating" I0512 11:37:05.673994 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5965dcf9f8-kb75q\" is waiting to start: ContainerCreating" I0512 11:37:05.674038 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5965dcf9f8-mlnmb pod in namespace openshift-image-registry (previous: false). I0512 11:37:05.873665 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5965dcf9f8-mlnmb pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5965dcf9f8-mlnmb\" is waiting to start: ContainerCreating" I0512 11:37:05.873692 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5965dcf9f8-mlnmb\" is waiting to start: ContainerCreating" I0512 11:37:05.873735 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-bb9f85ff7-2hdzq pod in namespace openshift-image-registry (previous: false). I0512 11:37:06.073029 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-bb9f85ff7-2hdzq pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-bb9f85ff7-2hdzq\" is waiting to start: ContainerCreating" I0512 11:37:06.073048 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-bb9f85ff7-2hdzq\" is waiting to start: ContainerCreating" I0512 11:37:06.073063 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-5588t pod in namespace openshift-image-registry (previous: false). I0512 11:37:06.272988 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 11:37:06.273005 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-828ws pod in namespace openshift-image-registry (previous: false). W0512 11:37:06.458382 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 11:37:06.475671 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 11:37:06.475688 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-dk26g pod in namespace openshift-image-registry (previous: false). I0512 11:37:06.673678 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 11:37:06.673698 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-65c856bc85-k42wv pod in namespace openshift-ingress (previous: false). I0512 11:37:06.872894 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-65c856bc85-k42wv pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-65c856bc85-k42wv\" is waiting to start: ContainerCreating" I0512 11:37:06.872912 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-65c856bc85-k42wv\" is waiting to start: ContainerCreating" I0512 11:37:06.872923 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-74f7964748-sfpw2 pod in namespace openshift-ingress (previous: false). I0512 11:37:07.080534 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-74f7964748-sfpw2 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-74f7964748-sfpw2\" is waiting to start: ContainerCreating" I0512 11:37:07.080558 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-74f7964748-sfpw2\" is waiting to start: ContainerCreating" I0512 11:37:07.080573 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-74f7964748-sg484 pod in namespace openshift-ingress (previous: false). I0512 11:37:07.279110 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-74f7964748-sg484 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-74f7964748-sg484\" is waiting to start: ContainerCreating" I0512 11:37:07.279129 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-74f7964748-sg484\" is waiting to start: ContainerCreating" I0512 11:37:07.279139 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-fpfn7 pod in namespace openshift-ingress-canary (previous: false). W0512 11:37:07.459574 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0512 11:37:07.459605 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 11:37:07.459626 1 tasks_processing.go:74] worker 23 stopped. E0512 11:37:07.459642 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 11:37:07.459656 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0512 11:37:07.459675 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0512 11:37:07.459689 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.045805902s to process 1 records I0512 11:37:07.479332 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-fpfn7 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fpfn7\" is waiting to start: ContainerCreating" I0512 11:37:07.479353 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fpfn7\" is waiting to start: ContainerCreating" I0512 11:37:07.479367 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-j2rsl pod in namespace openshift-ingress-canary (previous: false). I0512 11:37:07.672529 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-j2rsl pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-j2rsl\" is waiting to start: ContainerCreating" I0512 11:37:07.672549 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-j2rsl\" is waiting to start: ContainerCreating" I0512 11:37:07.672560 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-pzvjm pod in namespace openshift-ingress-canary (previous: false). I0512 11:37:07.884323 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-pzvjm pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-pzvjm\" is waiting to start: ContainerCreating" I0512 11:37:07.884341 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-pzvjm\" is waiting to start: ContainerCreating" I0512 11:37:07.884352 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-rn8gs pod in namespace openshift-kube-storage-version-migrator (previous: false). I0512 11:37:08.074147 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-rn8gs pod in namespace openshift-kube-storage-version-migrator (previous: false). I0512 11:37:08.272806 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-ch8mp pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0512 11:37:08.474544 1 tasks_processing.go:74] worker 39 stopped. I0512 11:37:08.474649 1 recorder.go:75] Recording events/openshift-dns with fingerprint=15b4000d3b3d49bf85268c754b70bceaa45bac4cc11952dd70c9aaa9a79cc876 I0512 11:37:08.474747 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=1b6af9eb9b6bc707bb4efebac00dfcae8b5d939764779d4c09cafb65274cd96c I0512 11:37:08.474785 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=a9590d7b84bc7ff9316a37036115898dfa0e431530308e6400f83762df415b41 I0512 11:37:08.474831 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=62269850e55c0bb5b650ff17b917674a9a58766a76e4c4fce302936b76911f72 I0512 11:37:08.474848 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=58a6ccbf53df9cf442eb31566781aca71f517ac216c420d228a8900091005563 I0512 11:37:08.474862 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=9eecec175c97eb4cce2e4e732a0c21f270b9ca44c72ba3a0c9270db5aed66d5c I0512 11:37:08.474910 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=18c21e8f7c5a161537d62605d9980da9f4340fc70847a2e486ae3c0772d25e22 I0512 11:37:08.476265 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-5965dcf9f8-kb75q with fingerprint=1988c4de22df623b7f2a2a8009d7ad796d73ba0bb5d092b6b7d65e342f22f722 I0512 11:37:08.476384 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-5965dcf9f8-mlnmb with fingerprint=a609cc92dfd180e46c76a230d27b3f6cddbf070b8bc8b24f57a6b6f8d35e0363 I0512 11:37:08.476475 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-bb9f85ff7-2hdzq with fingerprint=6cd25c9d31a889bbe357eb138a5f0a23cae9889b69cbdca80c489709176cfe4a I0512 11:37:08.476492 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-rn8gs/migrator_current.log with fingerprint=fdca3e3aeb156a1e60aba01dfec3c0ff8ce38c0c3094a77d609761b811d09053 I0512 11:37:08.476498 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-rn8gs/graceful-termination_current.log with fingerprint=086094361a26269d03062080a122c00e4ae6c182ee64239bb071041a941be5e9 I0512 11:37:08.476580 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-ch8mp/kube-storage-version-migrator-operator_current.log with fingerprint=1a818ad8d11569ee7f7283e048eb5f043d9c91f61883eaaa62dc5939158bac2c I0512 11:37:08.476592 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.060857297s to process 13 records I0512 11:37:15.266428 1 tasks_processing.go:74] worker 54 stopped. I0512 11:37:15.266464 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0512 11:37:15.266476 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.852662582s to process 1 records I0512 11:37:15.269120 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 11:37:16.019424 1 tasks_processing.go:74] worker 15 stopped. I0512 11:37:16.019700 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=9d475996b970b5499163b99cb7866431c26bc05ed77261e8b402c7515458063c I0512 11:37:16.019715 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.60746366s to process 1 records E0512 11:37:16.019768 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.608s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0512 11:37:16.020871 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0512 11:37:16.020885 1 periodic.go:209] Running workloads gatherer I0512 11:37:16.020898 1 tasks_processing.go:45] number of workers: 2 I0512 11:37:16.020906 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 11:37:16.020910 1 tasks_processing.go:71] worker 1 working on workload_info task. I0512 11:37:16.020924 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 11:37:16.020994 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0512 11:37:16.052662 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0512 11:37:16.067905 1 tasks_processing.go:74] worker 0 stopped. I0512 11:37:16.067922 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 46.891958ms to process 0 records I0512 11:37:16.068746 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (17ms) I0512 11:37:16.085586 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (17ms) I0512 11:37:16.105455 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (20ms) I0512 11:37:16.119712 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (14ms) I0512 11:37:16.133152 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (13ms) I0512 11:37:16.152045 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (19ms) I0512 11:37:16.167045 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (15ms) I0512 11:37:16.181475 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (14ms) I0512 11:37:16.196413 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (15ms) I0512 11:37:16.217235 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (21ms) I0512 11:37:16.229592 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (12ms) I0512 11:37:16.274207 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (45ms) I0512 11:37:16.350582 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 11:37:16.362481 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (88ms) I0512 11:37:16.462597 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0512 11:37:16.548894 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 11:37:16.562680 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0512 11:37:16.662881 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0512 11:37:16.762391 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (99ms) I0512 11:37:16.863119 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (101ms) I0512 11:37:16.962972 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0512 11:37:17.062410 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (99ms) I0512 11:37:17.162913 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0512 11:37:17.262475 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0512 11:37:17.362603 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0512 11:37:17.462141 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0512 11:37:17.562285 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0512 11:37:17.662737 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0512 11:37:17.763617 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (101ms) I0512 11:37:17.863393 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0512 11:37:17.962796 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (99ms) I0512 11:37:18.062152 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (99ms) I0512 11:37:18.162599 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0512 11:37:18.262865 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0512 11:37:18.363198 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (100ms) I0512 11:37:18.363228 1 tasks_processing.go:74] worker 1 stopped. E0512 11:37:18.363239 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0512 11:37:18.363577 1 recorder.go:75] Recording config/workload_info with fingerprint=ea79ffc413524c1d55cf8f630cf1fd852559bc0241009a1f465e2a5034a39ed0 I0512 11:37:18.363595 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.34231199s to process 1 records E0512 11:37:18.363619 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.342s with: function \"workload_info\" failed with an error" I0512 11:37:18.364719 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0512 11:37:18.364731 1 periodic.go:209] Running conditional gatherer I0512 11:37:18.372304 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0512 11:37:18.378684 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.10:33013->172.30.0.10:53: read: connection refused E0512 11:37:18.378918 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0512 11:37:18.378977 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0512 11:37:18.389561 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0512 11:37:18.389574 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389579 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389583 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389586 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389589 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389592 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389595 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389598 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 11:37:18.389601 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0512 11:37:18.389617 1 tasks_processing.go:45] number of workers: 3 I0512 11:37:18.389627 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 11:37:18.389630 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0512 11:37:18.389638 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 11:37:18.389658 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0512 11:37:18.389658 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 11:37:18.389664 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0512 11:37:18.389669 1 tasks_processing.go:74] worker 1 stopped. I0512 11:37:18.389714 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0512 11:37:18.389726 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.08µs to process 1 records I0512 11:37:18.389758 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0512 11:37:18.389767 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.115µs to process 1 records I0512 11:37:18.389773 1 tasks_processing.go:74] worker 0 stopped. I0512 11:37:18.389901 1 tasks_processing.go:74] worker 2 stopped. I0512 11:37:18.389913 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 229.348µs to process 0 records I0512 11:37:18.389935 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.10:33013->172.30.0.10:53: read: connection refused I0512 11:37:18.389953 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0512 11:37:18.411043 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=0b315c60fb2be8f648d4892461ee4095b4c1472e0838b845cb5685c9aec99af8 I0512 11:37:18.411155 1 diskrecorder.go:70] Writing 107 records to /var/lib/insights-operator/insights-2026-05-12-113718.tar.gz I0512 11:37:18.417904 1 diskrecorder.go:51] Wrote 107 records to disk in 6ms I0512 11:37:18.417934 1 periodic.go:278] Gathering cluster info every 2h0m0s I0512 11:37:18.417948 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0512 11:37:28.903529 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 11:38:21.751787 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="eb82f684e7b9396ced5b36b24c1d8019cffc99faa6ab7f7bc7877f8f0664b647") W0512 11:38:21.751816 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0512 11:38:21.751891 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="07798305ff6f6e932b85c3850c85a581235e376c90a21f9e76348c796b94e40b") I0512 11:38:21.751903 1 base_controller.go:181] Shutting down ConfigController ... I0512 11:38:21.751930 1 base_controller.go:181] Shutting down LoggingSyncer ... I0512 11:38:21.751947 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0512 11:38:21.751958 1 periodic.go:170] Shutting down I0512 11:38:21.751960 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0512 11:38:21.751971 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0512 11:38:21.751981 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="6aa9a68ca07e65ba487b4eaeda0e0de125db56f7f95bf8e180c768c013f0546d") I0512 11:38:21.751990 1 base_controller.go:113] All LoggingSyncer workers have been terminated E0512 11:38:21.751999 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled