W0505 23:19:09.372450 1 cmd.go:257] Using insecure, self-signed certificates I0505 23:19:09.995355 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0505 23:19:09.995665 1 observer_polling.go:159] Starting file observer I0505 23:19:10.965909 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0505 23:19:10.966118 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0505 23:19:10.966653 1 secure_serving.go:57] Forcing use of http/1.1 only W0505 23:19:10.966678 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0505 23:19:10.966683 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0505 23:19:10.966687 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0505 23:19:10.966689 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0505 23:19:10.966692 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0505 23:19:10.966694 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0505 23:19:10.966739 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0505 23:19:10.972058 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0505 23:19:10.972119 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"cecc487e-ed25-44ce-860d-de1b10325953", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0505 23:19:10.973985 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0505 23:19:10.974006 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0505 23:19:10.974010 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0505 23:19:10.974023 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0505 23:19:10.974029 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0505 23:19:10.974013 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0505 23:19:10.974260 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3757820443/tls.crt::/tmp/serving-cert-3757820443/tls.key" I0505 23:19:10.974481 1 secure_serving.go:213] Serving securely on [::]:8443 I0505 23:19:10.974509 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0505 23:19:10.980543 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0505 23:19:10.980564 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0505 23:19:10.980650 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0505 23:19:10.987563 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 23:19:10.987583 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0505 23:19:10.994215 1 secretconfigobserver.go:119] support secret does not exist I0505 23:19:11.001040 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 23:19:11.007737 1 secretconfigobserver.go:119] support secret does not exist I0505 23:19:11.011284 1 recorder.go:161] Pruning old reports every 8h24m3s, max age is 288h0m0s I0505 23:19:11.018480 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0505 23:19:11.018499 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0505 23:19:11.018502 1 periodic.go:209] Running clusterconfig gatherer I0505 23:19:11.018520 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0505 23:19:11.018525 1 insightsreport.go:296] Starting report retriever I0505 23:19:11.018530 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0505 23:19:11.018562 1 tasks_processing.go:45] number of workers: 64 I0505 23:19:11.018589 1 tasks_processing.go:69] worker 2 listening for tasks. I0505 23:19:11.018595 1 tasks_processing.go:69] worker 8 listening for tasks. I0505 23:19:11.018601 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 23:19:11.018604 1 tasks_processing.go:69] worker 3 listening for tasks. I0505 23:19:11.018609 1 tasks_processing.go:69] worker 5 listening for tasks. I0505 23:19:11.018609 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 23:19:11.018614 1 tasks_processing.go:69] worker 7 listening for tasks. I0505 23:19:11.018610 1 tasks_processing.go:69] worker 6 listening for tasks. I0505 23:19:11.018619 1 tasks_processing.go:69] worker 4 listening for tasks. I0505 23:19:11.018624 1 tasks_processing.go:69] worker 36 listening for tasks. I0505 23:19:11.018626 1 tasks_processing.go:69] worker 27 listening for tasks. I0505 23:19:11.018630 1 tasks_processing.go:69] worker 17 listening for tasks. I0505 23:19:11.018634 1 tasks_processing.go:69] worker 28 listening for tasks. I0505 23:19:11.018629 1 tasks_processing.go:69] worker 31 listening for tasks. I0505 23:19:11.018640 1 tasks_processing.go:69] worker 18 listening for tasks. I0505 23:19:11.018643 1 tasks_processing.go:69] worker 29 listening for tasks. I0505 23:19:11.018646 1 tasks_processing.go:69] worker 13 listening for tasks. I0505 23:19:11.018647 1 tasks_processing.go:69] worker 19 listening for tasks. I0505 23:19:11.018649 1 tasks_processing.go:69] worker 30 listening for tasks. I0505 23:19:11.018663 1 tasks_processing.go:69] worker 14 listening for tasks. I0505 23:19:11.018723 1 tasks_processing.go:71] worker 14 working on cost_management_metrics_configs task. I0505 23:19:11.018735 1 tasks_processing.go:69] worker 32 listening for tasks. I0505 23:19:11.018741 1 tasks_processing.go:71] worker 32 working on machine_autoscalers task. I0505 23:19:11.018752 1 tasks_processing.go:69] worker 49 listening for tasks. I0505 23:19:11.018764 1 tasks_processing.go:69] worker 43 listening for tasks. I0505 23:19:11.018770 1 tasks_processing.go:69] worker 44 listening for tasks. I0505 23:19:11.018766 1 tasks_processing.go:69] worker 26 listening for tasks. I0505 23:19:11.018772 1 tasks_processing.go:69] worker 9 listening for tasks. I0505 23:19:11.018775 1 tasks_processing.go:69] worker 45 listening for tasks. I0505 23:19:11.018783 1 tasks_processing.go:69] worker 10 listening for tasks. I0505 23:19:11.018775 1 tasks_processing.go:69] worker 42 listening for tasks. I0505 23:19:11.018791 1 tasks_processing.go:69] worker 33 listening for tasks. I0505 23:19:11.018789 1 tasks_processing.go:69] worker 34 listening for tasks. I0505 23:19:11.018799 1 tasks_processing.go:69] worker 38 listening for tasks. I0505 23:19:11.018807 1 tasks_processing.go:69] worker 41 listening for tasks. I0505 23:19:11.018805 1 tasks_processing.go:69] worker 48 listening for tasks. I0505 23:19:11.018811 1 tasks_processing.go:69] worker 35 listening for tasks. I0505 23:19:11.018810 1 tasks_processing.go:69] worker 39 listening for tasks. I0505 23:19:11.018804 1 tasks_processing.go:69] worker 16 listening for tasks. I0505 23:19:11.018803 1 tasks_processing.go:69] worker 15 listening for tasks. I0505 23:19:11.018821 1 tasks_processing.go:69] worker 22 listening for tasks. I0505 23:19:11.018825 1 tasks_processing.go:71] worker 28 working on schedulers task. I0505 23:19:11.018814 1 tasks_processing.go:69] worker 11 listening for tasks. I0505 23:19:11.018831 1 tasks_processing.go:69] worker 52 listening for tasks. I0505 23:19:11.018831 1 tasks_processing.go:71] worker 13 working on version task. I0505 23:19:11.018833 1 tasks_processing.go:69] worker 53 listening for tasks. I0505 23:19:11.018836 1 tasks_processing.go:71] worker 19 working on cluster_apiserver task. I0505 23:19:11.018829 1 tasks_processing.go:69] worker 51 listening for tasks. I0505 23:19:11.018835 1 tasks_processing.go:71] worker 29 working on feature_gates task. I0505 23:19:11.018839 1 tasks_processing.go:71] worker 4 working on machine_sets task. I0505 23:19:11.018844 1 tasks_processing.go:71] worker 36 working on openstack_dataplanenodesets task. I0505 23:19:11.018847 1 tasks_processing.go:69] worker 62 listening for tasks. I0505 23:19:11.018851 1 tasks_processing.go:71] worker 8 working on overlapping_namespace_uids task. I0505 23:19:11.018849 1 tasks_processing.go:71] worker 17 working on proxies task. I0505 23:19:11.018851 1 tasks_processing.go:71] worker 2 working on sap_datahubs task. I0505 23:19:11.018857 1 tasks_processing.go:69] worker 63 listening for tasks. I0505 23:19:11.018843 1 tasks_processing.go:71] worker 30 working on ingress task. I0505 23:19:11.018836 1 tasks_processing.go:71] worker 0 working on nodenetworkconfigurationpolicies task. I0505 23:19:11.018955 1 tasks_processing.go:71] worker 63 working on service_accounts task. I0505 23:19:11.018964 1 tasks_processing.go:71] worker 41 working on node_logs task. I0505 23:19:11.018826 1 tasks_processing.go:71] worker 3 working on aggregated_monitoring_cr_names task. I0505 23:19:11.019560 1 tasks_processing.go:69] worker 23 listening for tasks. I0505 23:19:11.019571 1 tasks_processing.go:71] worker 23 working on config_maps task. I0505 23:19:11.019604 1 tasks_processing.go:71] worker 33 working on storage_cluster task. I0505 23:19:11.019707 1 tasks_processing.go:71] worker 39 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0505 23:19:11.019809 1 tasks_processing.go:71] worker 48 working on validating_webhook_configurations task. I0505 23:19:11.019832 1 tasks_processing.go:71] worker 10 working on certificate_signing_requests task. I0505 23:19:11.020156 1 tasks_processing.go:71] worker 42 working on jaegers task. I0505 23:19:11.018803 1 tasks_processing.go:69] worker 40 listening for tasks. I0505 23:19:11.018786 1 tasks_processing.go:69] worker 46 listening for tasks. I0505 23:19:11.018814 1 tasks_processing.go:69] worker 50 listening for tasks. I0505 23:19:11.018816 1 tasks_processing.go:69] worker 58 listening for tasks. I0505 23:19:11.018791 1 tasks_processing.go:69] worker 37 listening for tasks. I0505 23:19:11.018820 1 tasks_processing.go:69] worker 54 listening for tasks. I0505 23:19:11.018819 1 tasks_processing.go:71] worker 27 working on support_secret task. I0505 23:19:11.018822 1 tasks_processing.go:69] worker 59 listening for tasks. I0505 23:19:11.018807 1 tasks_processing.go:69] worker 21 listening for tasks. I0505 23:19:11.018822 1 tasks_processing.go:71] worker 18 working on dvo_metrics task. I0505 23:19:11.018818 1 tasks_processing.go:69] worker 24 listening for tasks. I0505 23:19:11.018819 1 tasks_processing.go:69] worker 25 listening for tasks. I0505 23:19:11.018828 1 tasks_processing.go:69] worker 55 listening for tasks. I0505 23:19:11.021391 1 tasks_processing.go:71] worker 59 working on networks task. I0505 23:19:11.021395 1 tasks_processing.go:71] worker 21 working on clusterroles task. I0505 23:19:11.018832 1 tasks_processing.go:71] worker 5 working on mutating_webhook_configurations task. I0505 23:19:11.018828 1 tasks_processing.go:71] worker 31 working on openshift_machine_api_events task. I0505 23:19:11.018831 1 tasks_processing.go:69] worker 20 listening for tasks. I0505 23:19:11.018839 1 tasks_processing.go:71] worker 6 working on nodes task. I0505 23:19:11.018838 1 tasks_processing.go:69] worker 61 listening for tasks. I0505 23:19:11.018836 1 tasks_processing.go:69] worker 56 listening for tasks. I0505 23:19:11.018834 1 tasks_processing.go:69] worker 12 listening for tasks. I0505 23:19:11.018842 1 tasks_processing.go:71] worker 7 working on pod_network_connectivity_checks task. I0505 23:19:11.018842 1 tasks_processing.go:69] worker 57 listening for tasks. I0505 23:19:11.018794 1 tasks_processing.go:69] worker 47 listening for tasks. I0505 23:19:11.018968 1 tasks_processing.go:71] worker 22 working on pdbs task. I0505 23:19:11.018967 1 tasks_processing.go:71] worker 45 working on openstack_version task. I0505 23:19:11.018969 1 tasks_processing.go:71] worker 11 working on install_plans task. I0505 23:19:11.020316 1 tasks_processing.go:71] worker 34 working on silenced_alerts task. I0505 23:19:11.018805 1 tasks_processing.go:71] worker 1 working on machine_config_pools task. I0505 23:19:11.020317 1 tasks_processing.go:71] worker 44 working on machine_healthchecks task. I0505 23:19:11.020394 1 tasks_processing.go:71] worker 26 working on container_images task. I0505 23:19:11.020402 1 tasks_processing.go:71] worker 49 working on qemu_kubevirt_launcher_logs task. I0505 23:19:11.020412 1 tasks_processing.go:71] worker 43 working on image task. I0505 23:19:11.020464 1 tasks_processing.go:71] worker 53 working on openstack_controlplanes task. I0505 23:19:11.020466 1 tasks_processing.go:71] worker 51 working on machine_configs task. I0505 23:19:11.020472 1 tasks_processing.go:71] worker 15 working on operators_pods_and_events task. I0505 23:19:11.020496 1 tasks_processing.go:71] worker 9 working on sap_config task. I0505 23:19:11.020520 1 tasks_processing.go:71] worker 52 working on monitoring_persistent_volumes task. I0505 23:19:11.020548 1 tasks_processing.go:71] worker 38 working on authentication task. I0505 23:19:11.020560 1 tasks_processing.go:71] worker 16 working on crds task. I0505 23:19:11.020587 1 tasks_processing.go:71] worker 62 working on infrastructures task. I0505 23:19:11.020872 1 tasks_processing.go:71] worker 35 working on tsdb_status task. I0505 23:19:11.021266 1 tasks_processing.go:71] worker 40 working on sap_pods task. I0505 23:19:11.022034 1 tasks_processing.go:71] worker 47 working on image_pruners task. I0505 23:19:11.022149 1 tasks_processing.go:71] worker 20 working on ceph_cluster task. W0505 23:19:11.022220 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0505 23:19:11.022284 1 tasks_processing.go:71] worker 35 working on active_alerts task. I0505 23:19:11.022296 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 78.02µs to process 0 records W0505 23:19:11.022311 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0505 23:19:11.021281 1 tasks_processing.go:71] worker 37 working on openstack_dataplanedeployments task. I0505 23:19:11.022326 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 20.38µs to process 0 records I0505 23:19:11.022329 1 tasks_processing.go:71] worker 35 working on openshift_logging task. I0505 23:19:11.022393 1 tasks_processing.go:71] worker 61 working on container_runtime_configs task. I0505 23:19:11.021286 1 tasks_processing.go:71] worker 46 working on metrics task. I0505 23:19:11.022445 1 tasks_processing.go:71] worker 56 working on olm_operators task. W0505 23:19:11.022455 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0505 23:19:11.021290 1 tasks_processing.go:71] worker 58 working on machines task. I0505 23:19:11.022484 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 34.814µs to process 0 records I0505 23:19:11.022459 1 tasks_processing.go:71] worker 12 working on lokistack task. I0505 23:19:11.021317 1 tasks_processing.go:71] worker 54 working on image_registries task. I0505 23:19:11.018831 1 tasks_processing.go:69] worker 60 listening for tasks. I0505 23:19:11.021293 1 tasks_processing.go:71] worker 50 working on oauths task. I0505 23:19:11.022614 1 tasks_processing.go:74] worker 60 stopped. I0505 23:19:11.021441 1 tasks_processing.go:71] worker 55 working on nodenetworkstates task. I0505 23:19:11.021440 1 tasks_processing.go:71] worker 25 working on operators task. I0505 23:19:11.021443 1 tasks_processing.go:71] worker 24 working on storage_classes task. W0505 23:19:11.021639 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0505 23:19:11.022739 1 tasks_processing.go:74] worker 34 stopped. I0505 23:19:11.022750 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.116184ms to process 0 records I0505 23:19:11.022561 1 tasks_processing.go:71] worker 57 working on ingress_certificates task. I0505 23:19:11.022478 1 tasks_processing.go:74] worker 46 stopped. I0505 23:19:11.023600 1 tasks_processing.go:74] worker 36 stopped. I0505 23:19:11.023612 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 4.743716ms to process 0 records I0505 23:19:11.023629 1 tasks_processing.go:74] worker 14 stopped. I0505 23:19:11.023633 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 4.898101ms to process 0 records I0505 23:19:11.023841 1 tasks_processing.go:74] worker 4 stopped. I0505 23:19:11.023854 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 4.986961ms to process 0 records I0505 23:19:11.027537 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0505 23:19:11.027553 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0505 23:19:11.027558 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0505 23:19:11.027561 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0505 23:19:11.027577 1 controller.go:489] The operator is still being initialized I0505 23:19:11.027584 1 controller.go:512] The operator is healthy I0505 23:19:11.027706 1 tasks_processing.go:74] worker 32 stopped. I0505 23:19:11.027718 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 8.953333ms to process 0 records I0505 23:19:11.031817 1 tasks_processing.go:74] worker 2 stopped. I0505 23:19:11.031831 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 12.944699ms to process 0 records I0505 23:19:11.032160 1 tasks_processing.go:74] worker 0 stopped. I0505 23:19:11.032169 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 13.222657ms to process 0 records I0505 23:19:11.032186 1 tasks_processing.go:74] worker 33 stopped. I0505 23:19:11.032196 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 12.569485ms to process 0 records I0505 23:19:11.032587 1 tasks_processing.go:74] worker 61 stopped. I0505 23:19:11.032600 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 10.176072ms to process 0 records I0505 23:19:11.035712 1 tasks_processing.go:74] worker 42 stopped. I0505 23:19:11.035725 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 14.450184ms to process 0 records I0505 23:19:11.036066 1 tasks_processing.go:74] worker 53 stopped. I0505 23:19:11.036079 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 14.138609ms to process 0 records I0505 23:19:11.036186 1 tasks_processing.go:74] worker 44 stopped. E0505 23:19:11.036198 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0505 23:19:11.036205 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 14.55642ms to process 0 records I0505 23:19:11.037719 1 tasks_processing.go:74] worker 12 stopped. I0505 23:19:11.037729 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 15.224208ms to process 0 records I0505 23:19:11.037735 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 16.158788ms to process 0 records I0505 23:19:11.037742 1 tasks_processing.go:74] worker 45 stopped. I0505 23:19:11.037774 1 tasks_processing.go:74] worker 58 stopped. E0505 23:19:11.037783 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0505 23:19:11.037790 1 gather.go:177] gatherer "clusterconfig" function "machines" took 15.289988ms to process 0 records I0505 23:19:11.037976 1 tasks_processing.go:74] worker 19 stopped. I0505 23:19:11.038175 1 recorder.go:75] Recording config/apiserver with fingerprint=d7a78a0e409c6c2a423027a072f6dd7f26a2e3d61d511bd151efe3acc8daccc2 I0505 23:19:11.038192 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 19.124696ms to process 1 records I0505 23:19:11.038205 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 16.17019ms to process 0 records I0505 23:19:11.038214 1 tasks_processing.go:74] worker 40 stopped. I0505 23:19:11.041808 1 tasks_processing.go:74] worker 31 stopped. I0505 23:19:11.041824 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 20.338108ms to process 0 records I0505 23:19:11.041919 1 tasks_processing.go:74] worker 43 stopped. I0505 23:19:11.042044 1 recorder.go:75] Recording config/image with fingerprint=a196393ceb0e964525db72b0c0dfd39be4a09fbfa10a94e7354b4c6a179d392b I0505 23:19:11.042060 1 gather.go:177] gatherer "clusterconfig" function "image" took 20.276979ms to process 1 records I0505 23:19:11.042262 1 tasks_processing.go:74] worker 17 stopped. I0505 23:19:11.042330 1 recorder.go:75] Recording config/proxy with fingerprint=92cc1373ded68e749dbab58599386bc987ebf8dd1ca8f22b258d9b17dea2287d I0505 23:19:11.042343 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 23.394291ms to process 1 records I0505 23:19:11.042427 1 tasks_processing.go:74] worker 29 stopped. I0505 23:19:11.042461 1 recorder.go:75] Recording config/featuregate with fingerprint=efd054f846c86d5239d402fb3620683888ff0178d29c1c2167c3733ff63312cd I0505 23:19:11.042471 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 23.412098ms to process 1 records I0505 23:19:11.042550 1 tasks_processing.go:74] worker 59 stopped. I0505 23:19:11.042582 1 recorder.go:75] Recording config/network with fingerprint=cbfaa8f6eb0506503cd87c860cbc6c4f5e3758ee6e52bfb7f62ef20da0c70c39 I0505 23:19:11.042590 1 gather.go:177] gatherer "clusterconfig" function "networks" took 20.910463ms to process 1 records I0505 23:19:11.046931 1 tasks_processing.go:74] worker 7 stopped. E0505 23:19:11.046955 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0505 23:19:11.046966 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 25.386337ms to process 0 records I0505 23:19:11.046977 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 24.767628ms to process 0 records I0505 23:19:11.046985 1 tasks_processing.go:74] worker 20 stopped. I0505 23:19:11.047199 1 tasks_processing.go:74] worker 55 stopped. I0505 23:19:11.047208 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 24.575045ms to process 0 records I0505 23:19:11.047218 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 24.876071ms to process 0 records I0505 23:19:11.047225 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 24.874735ms to process 0 records I0505 23:19:11.047229 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 25.017872ms to process 0 records I0505 23:19:11.047233 1 tasks_processing.go:74] worker 9 stopped. I0505 23:19:11.047236 1 tasks_processing.go:74] worker 37 stopped. I0505 23:19:11.047240 1 tasks_processing.go:74] worker 35 stopped. I0505 23:19:11.055397 1 tasks_processing.go:74] worker 47 stopped. I0505 23:19:11.055683 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=755bf4afcc4ec20a0a7c4d25518f1cc8342832094979fb9b807953bdc1a3ea45 I0505 23:19:11.055698 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 33.352407ms to process 1 records E0505 23:19:11.055714 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0505 23:19:11.055726 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 34.186091ms to process 0 records I0505 23:19:11.055728 1 tasks_processing.go:74] worker 27 stopped. I0505 23:19:11.055802 1 tasks_processing.go:74] worker 62 stopped. I0505 23:19:11.056431 1 recorder.go:75] Recording config/infrastructure with fingerprint=c6b87dfb526506352c526d18b5dda44e7f394354a95c8195ad56312f5aec2819 I0505 23:19:11.056690 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 33.532283ms to process 1 records I0505 23:19:11.056848 1 tasks_processing.go:74] worker 50 stopped. I0505 23:19:11.057290 1 recorder.go:75] Recording config/oauth with fingerprint=7ca60993df3b9037ab040ff8dc1545acd3b22a62ad8a07abf4bb1bf03c9bab20 I0505 23:19:11.057358 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 33.49773ms to process 1 records I0505 23:19:11.057510 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=a6b3b39265fe345851148bf2674453ec3cbebb2c8c1e3a674e3fb3f2dfe278b7 I0505 23:19:11.057540 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 38.015266ms to process 1 records I0505 23:19:11.057548 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 35.490062ms to process 0 records I0505 23:19:11.057585 1 tasks_processing.go:74] worker 28 stopped. I0505 23:19:11.057601 1 tasks_processing.go:74] worker 1 stopped. I0505 23:19:11.057667 1 tasks_processing.go:74] worker 30 stopped. I0505 23:19:11.057770 1 recorder.go:75] Recording config/ingress with fingerprint=5e2e15dfd1e1f7a1ed19cfe7e411937b2fc22d194a54d0c15813b2ccdfa772bb I0505 23:19:11.057812 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 38.572223ms to process 1 records I0505 23:19:11.060483 1 tasks_processing.go:74] worker 52 stopped. I0505 23:19:11.060499 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 38.266862ms to process 0 records I0505 23:19:11.060514 1 tasks_processing.go:74] worker 22 stopped. I0505 23:19:11.060609 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=5c0b7360369a6ca99eac45798ff7e85d6e2bdf1c9d33f1f0963e9246a00f19e6 I0505 23:19:11.060634 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=278321e9a65c3e4c56a9d1cd3e727fd0bba4dd7241a550d3857e1612df7a0a63 I0505 23:19:11.060649 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=544eb02db3bafb9870cfc6b190e1b0b48dc8e7c024ae8bd441cbcdad1a13b6ea I0505 23:19:11.060656 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 38.886476ms to process 3 records I0505 23:19:11.061012 1 tasks_processing.go:74] worker 41 stopped. I0505 23:19:11.061031 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 41.564622ms to process 0 records I0505 23:19:11.061114 1 tasks_processing.go:74] worker 5 stopped. I0505 23:19:11.061282 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=fc2da9f70b9a1c3d7deab0d4dfafb3d5c55c91bafc1ef9dee756fd8e33a3a557 I0505 23:19:11.061318 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=6f2760af7d587d265df73f9eaa05395d9c7ca1aa97b7aa9caa0bd976640967d8 I0505 23:19:11.061348 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=5a40ff07d709dd18fdd63ebb927db046729887afc165bc690c041ae87ccf1782 I0505 23:19:11.061357 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 39.661369ms to process 3 records I0505 23:19:11.061436 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=023c47b4ffc4f908c16ba4835076df8f67d13c71d781d7278b46a1907d00cd71 I0505 23:19:11.061439 1 tasks_processing.go:74] worker 48 stopped. I0505 23:19:11.061511 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=1173ac37873d733fabf95efa6e7b7eaafda6949ec7e331b41fd80e6fb00901e3 I0505 23:19:11.061529 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=c3af1b696902d6ab586bd1e191d73d2e6f6ceb642c22ab9e86bdaea59c0a1636 I0505 23:19:11.061555 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=2165c56493de3a82803ec41f01b8069856b6a0cafa9a102f180ba6963d5c8e67 I0505 23:19:11.061578 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=87ac742fb5f2eea783fde94601b24e96fcd89fd687ce6373e14c23907876cad1 I0505 23:19:11.061603 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=6515eb67b0df6898c4101a451ecfceda4b355246e46266eca5fd675fd108149f I0505 23:19:11.061627 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=e7e655365a4bad4ff07d4ee7596753bb061ad24d352ff45093a3bf12f1147a12 I0505 23:19:11.061661 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=b18bb613dcad17b1fc6c46e1cd56569c04e6cc6d034ae2d15a120828f131980f I0505 23:19:11.061685 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=1a0855a994cdc15a59b94dd70eb928566b0381550b129a1f37ab625ec684869c I0505 23:19:11.061708 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=1738bfc9f92599c5e03424477a3da3c0e60cf17b588398e36440d5962d580884 I0505 23:19:11.061733 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=f30120bc14483bf80bef6e6511552bb267f25b365719cc658f623d4135b9d3db I0505 23:19:11.061741 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 41.300101ms to process 11 records I0505 23:19:11.061972 1 tasks_processing.go:74] worker 8 stopped. I0505 23:19:11.062002 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0505 23:19:11.062016 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 43.103297ms to process 1 records I0505 23:19:11.063384 1 tasks_processing.go:74] worker 24 stopped. I0505 23:19:11.063677 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=cd11d853680c167dea661aad034a765833c415b1d56b89ad00c9e64c06dee593 I0505 23:19:11.063714 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=1a637a41eba8c7b5c303d3b97275ac82ca5698070f70937f390be0c2cf29a531 I0505 23:19:11.063722 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 40.697964ms to process 2 records W0505 23:19:11.064534 1 operator.go:288] started I0505 23:19:11.064568 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0505 23:19:11.064774 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0505 23:19:11.064912 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0505 23:19:11.064976 1 tasks_processing.go:74] worker 6 stopped. I0505 23:19:11.066526 1 recorder.go:75] Recording config/node/ip-10-0-0-120.ec2.internal with fingerprint=931438bfda5e7a24ace571664eac7478e5f7913b059af561a6f4015010e66b6d I0505 23:19:11.066633 1 gather_logs.go:145] no pods in namespace were found I0505 23:19:11.066849 1 recorder.go:75] Recording config/node/ip-10-0-1-77.ec2.internal with fingerprint=376feadf93acac76f5a785f5340bff7c19f82da943c0e62976b2b2fe0e3aba63 I0505 23:19:11.067107 1 recorder.go:75] Recording config/node/ip-10-0-2-33.ec2.internal with fingerprint=9eb8f827b5f162557e3b7989bd1f521453b537c4bb422bf869eff42ed53be2db I0505 23:19:11.067203 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 43.443726ms to process 3 records I0505 23:19:11.067822 1 tasks_processing.go:74] worker 54 stopped. I0505 23:19:11.068260 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=16576807378ed2fb478ce53ee38ee7791ae5434bb3a33b2178fe36f540e4a995 I0505 23:19:11.068277 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 43.817284ms to process 1 records I0505 23:19:11.068371 1 tasks_processing.go:74] worker 38 stopped. I0505 23:19:11.068429 1 recorder.go:75] Recording config/authentication with fingerprint=fe01efaf47fc5c6aa1babdbfc1aeea648c4df9ddd2a820a043cfeecc1272d3ab I0505 23:19:11.068438 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 44.610973ms to process 1 records I0505 23:19:11.068444 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 44.74093ms to process 0 records I0505 23:19:11.068477 1 tasks_processing.go:74] worker 49 stopped. I0505 23:19:11.068547 1 tasks_processing.go:74] worker 21 stopped. I0505 23:19:11.068630 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=8b2788e87df1dd18a488059445194d62e5795d426e8473e2d2e0e45ffb534918 I0505 23:19:11.068718 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=8ba7313846b94cbe916f592b33ac0508af4e0c8e62c6a4bc23bfe8ab89c9374c I0505 23:19:11.068725 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 46.553852ms to process 2 records I0505 23:19:11.069011 1 tasks_processing.go:74] worker 3 stopped. I0505 23:19:11.069032 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 49.475161ms to process 0 records I0505 23:19:11.069360 1 tasks_processing.go:74] worker 10 stopped. I0505 23:19:11.069374 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 49.505643ms to process 0 records W0505 23:19:11.072574 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 23:19:11.074639 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0505 23:19:11.074746 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0505 23:19:11.074772 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0505 23:19:11.078808 1 tasks_processing.go:74] worker 16 stopped. I0505 23:19:11.079372 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=b9fbb2790b01ff345a88d78d33b220a6e94c162e7349a38c6106af9a27942b4c I0505 23:19:11.079582 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=2a61b5474cd5e7765cccf460ed1d1a5799941c93ca7b12b136429255db58340b I0505 23:19:11.079592 1 gather.go:177] gatherer "clusterconfig" function "crds" took 56.664517ms to process 2 records I0505 23:19:11.079639 1 recorder.go:75] Recording config/olm_operators with fingerprint=b163eed755fd451e07cd10d7c49c23924b37ecc5f90552df60ee0eff25e5ea1a I0505 23:19:11.079650 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 56.487803ms to process 1 records I0505 23:19:11.079659 1 tasks_processing.go:74] worker 56 stopped. I0505 23:19:11.080793 1 base_controller.go:82] Caches are synced for ConfigController I0505 23:19:11.080804 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0505 23:19:11.080890 1 tasks_processing.go:74] worker 39 stopped. I0505 23:19:11.080905 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 61.167535ms to process 0 records I0505 23:19:11.084596 1 controller.go:212] Source scaController *sca.Controller is not ready I0505 23:19:11.084610 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0505 23:19:11.084614 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0505 23:19:11.084616 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0505 23:19:11.084619 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0505 23:19:11.084635 1 controller.go:489] The operator is still being initialized I0505 23:19:11.084641 1 controller.go:512] The operator is healthy I0505 23:19:11.087632 1 prometheus_rules.go:88] Prometheus rules successfully created I0505 23:19:11.088803 1 tasks_processing.go:74] worker 13 stopped. I0505 23:19:11.089149 1 recorder.go:75] Recording config/version with fingerprint=7640f619f2aa2b479c00e39a86b8ee4831b6a4e25d05cbe00f67c0186385c166 I0505 23:19:11.089166 1 recorder.go:75] Recording config/id with fingerprint=9b468a5a4c2a7bbafb8450efeb7c19f9072c2489239f833aff8f3da9254d2a23 I0505 23:19:11.089173 1 gather.go:177] gatherer "clusterconfig" function "version" took 69.953268ms to process 2 records I0505 23:19:11.096330 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 23:19:11.104903 1 tasks_processing.go:74] worker 26 stopped. I0505 23:19:11.106313 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-7sjww with fingerprint=3bb874cfe953a11aaaf3ac529c552ae9c204736e1a2dc33fc8e0484807e51ffe I0505 23:19:11.106372 1 recorder.go:75] Recording config/running_containers with fingerprint=f9b62c23594e1afc8f38d282e8f2a727dae6f0a9a1c8e6bce13684302d1fecd1 I0505 23:19:11.106382 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 83.264824ms to process 2 records E0505 23:19:11.106487 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27496ae723-3bd4-42a8-aa02-96fe73a119b5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.15:38670->172.30.0.10:53: read: connection refused I0505 23:19:11.106500 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27496ae723-3bd4-42a8-aa02-96fe73a119b5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.15:38670->172.30.0.10:53: read: connection refused I0505 23:19:11.123665 1 tasks_processing.go:74] worker 23 stopped. E0505 23:19:11.123686 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0505 23:19:11.123693 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0505 23:19:11.123696 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0505 23:19:11.123710 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0505 23:19:11.123738 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0505 23:19:11.123745 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0505 23:19:11.123750 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0505 23:19:11.123755 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0505 23:19:11.123795 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0505 23:19:11.123802 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0505 23:19:11.123807 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 104.074622ms to process 7 records I0505 23:19:11.158458 1 tasks_processing.go:74] worker 51 stopped. I0505 23:19:11.158504 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0505 23:19:11.158521 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 136.430858ms to process 1 records I0505 23:19:11.165615 1 base_controller.go:82] Caches are synced for LoggingSyncer I0505 23:19:11.165632 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0505 23:19:11.170232 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0505 23:19:11.173304 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found W0505 23:19:11.174120 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.15:55864->172.30.0.10:53: read: connection refused I0505 23:19:11.174135 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.15:55864->172.30.0.10:53: read: connection refused I0505 23:19:11.193226 1 tasks_processing.go:74] worker 57 stopped. E0505 23:19:11.193246 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0505 23:19:11.193253 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q42fl5um0ac787k2dpdmuh4vo7mjhnq-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q42fl5um0ac787k2dpdmuh4vo7mjhnq-primary-cert-bundle-secret" not found I0505 23:19:11.193327 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=a5ebb115e8410f33b5d355662a18c091ece5f8cfd6e7d481f665fed722426659 I0505 23:19:11.193343 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 170.455049ms to process 1 records W0505 23:19:12.070678 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 23:19:12.277331 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0505 23:19:12.277348 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0505 23:19:12.278094 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-9mlvg pod in namespace openshift-dns (previous: false). I0505 23:19:12.494320 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0505 23:19:12.508280 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-9mlvg pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-9mlvg\" is waiting to start: ContainerCreating" I0505 23:19:12.508298 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-9mlvg\" is waiting to start: ContainerCreating" I0505 23:19:12.508305 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-9mlvg pod in namespace openshift-dns (previous: false). I0505 23:19:12.686716 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-9mlvg pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-9mlvg\" is waiting to start: ContainerCreating" I0505 23:19:12.686733 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-9mlvg\" is waiting to start: ContainerCreating" I0505 23:19:12.686774 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-bpb4w pod in namespace openshift-dns (previous: false). I0505 23:19:12.911824 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-bpb4w pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-bpb4w\" is waiting to start: ContainerCreating" I0505 23:19:12.911844 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-bpb4w\" is waiting to start: ContainerCreating" I0505 23:19:12.911856 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-bpb4w pod in namespace openshift-dns (previous: false). W0505 23:19:13.071732 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 23:19:13.098767 1 tasks_processing.go:74] worker 25 stopped. I0505 23:19:13.098835 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=da2f882a6eddcad47fe1e2788665cfbee8f55836ae7df3ba6954b51d1744cefb I0505 23:19:13.098868 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=6fdde9de0c0340b0d2860ba1dddee09d3c9136eff8bb224f70622c9436103013 I0505 23:19:13.098897 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0505 23:19:13.098925 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=ba1ea29ae98eb74fb4703dad4deb2b5f141f93286902ca671aae6008db4463b0 I0505 23:19:13.098989 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0505 23:19:13.099013 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=ff92a0239e57664a4187fa3ad296b8802458bbe6af1177572b4aa078ba3161b4 I0505 23:19:13.099043 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=661866ed82fd5a89496ec8ed8484351edae0b8669b8605feea1b6bce8ac1b819 I0505 23:19:13.099069 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=2ef98afc3a2c727e89eb20ec06a24e76c1c432d1272d67c09227c49f9b7d04b7 I0505 23:19:13.099084 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=a1b3814691edc5d80cf541f68436440d818754453f556e7b8858253d2f4ad75b I0505 23:19:13.099102 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=7685c264742bcb4d9dfe19d8ad5e4d3a3798ff0efd0201692d22b56589248f92 I0505 23:19:13.099112 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0505 23:19:13.099128 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=9c8ed4268abe5adadb0365b9ce26dec0e70da9fb80d4fa1d43c2ebc8cfb15c1f I0505 23:19:13.099140 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0505 23:19:13.099157 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=2bfbff8bd515a69305bbfa3e503c0df89378c7d51b61f299f9e3fe4c3d0e8abe I0505 23:19:13.099168 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0505 23:19:13.099187 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=6c95859c2db5d3c7a1949df625c6f21f7da4e567a060379361e6dfaf125a47ba I0505 23:19:13.099197 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0505 23:19:13.099213 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=bbd00e7882064ba7d31b8cedd293dc98abc8341b7e33b91bf0d2ca90cf35c1f3 I0505 23:19:13.099341 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=e763786661a890e79697eee78d3e669b47b28b8c469c89511d367c268747fca0 I0505 23:19:13.099352 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0505 23:19:13.099359 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0505 23:19:13.099381 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0505 23:19:13.099405 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=442908007be972529e5881d460f4b354f04a405e55b33eb4c2bc3411b0403c71 I0505 23:19:13.099429 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=20965de7a0d38df8fada8183f6a6073446d089b26062a024d224ec33ed96ea6e I0505 23:19:13.099439 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0505 23:19:13.099455 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=9a571942b04e3292d73c8364da135bc61356dcd40e24e0bb5df8f06f327af0d6 I0505 23:19:13.099465 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0505 23:19:13.099479 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=ca62454ac64b710012cc45a22533ef0fddace51b2065a39770da09731b809a1b I0505 23:19:13.099494 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=b00850ebb4114a7245ecec88ff7543926aa10b49ff6383bc516488eb385452db I0505 23:19:13.099510 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=ba1ff65ebce26c85bb264947ed69743a4d7db6dbc2930d04d6451156eef61b5c I0505 23:19:13.099522 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=2a349c68e06685d580bb7a57425ceffed313742c1bc8fb7d839437ebab106ce8 I0505 23:19:13.099543 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=c7c5a85e8890ebdbeecb9083f3add2a12fab3f9225bae6d5c8ba7e8b2fc80ff6 I0505 23:19:13.099552 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0505 23:19:13.099579 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=a94abd4c1c6460341d61528363e959308155a74003a2b2c04e60d4cf9a46079f I0505 23:19:13.099595 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0505 23:19:13.099604 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0505 23:19:13.099611 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.076094859s to process 36 records I0505 23:19:13.101796 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-bpb4w pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-bpb4w\" is waiting to start: ContainerCreating" I0505 23:19:13.101813 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-bpb4w\" is waiting to start: ContainerCreating" I0505 23:19:13.101841 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-t6tb8 pod in namespace openshift-dns (previous: false). I0505 23:19:13.304735 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-t6tb8 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-t6tb8\" is waiting to start: ContainerCreating" I0505 23:19:13.304756 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-t6tb8\" is waiting to start: ContainerCreating" I0505 23:19:13.304765 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-t6tb8 pod in namespace openshift-dns (previous: false). I0505 23:19:13.482574 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-t6tb8 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-t6tb8\" is waiting to start: ContainerCreating" I0505 23:19:13.482594 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-t6tb8\" is waiting to start: ContainerCreating" I0505 23:19:13.482604 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-7tx6w pod in namespace openshift-dns (previous: false). I0505 23:19:13.684342 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 23:19:13.684362 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-pnwvx pod in namespace openshift-dns (previous: false). I0505 23:19:13.882656 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 23:19:13.882675 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-xkz9h pod in namespace openshift-dns (previous: false). W0505 23:19:14.071372 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 23:19:14.085338 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 23:19:14.085386 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-746546b4d6-fsxkc pod in namespace openshift-image-registry (previous: false). I0505 23:19:14.282120 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-746546b4d6-fsxkc pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-746546b4d6-fsxkc\" is waiting to start: ContainerCreating" I0505 23:19:14.282138 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-746546b4d6-fsxkc\" is waiting to start: ContainerCreating" I0505 23:19:14.282171 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-746546b4d6-lx4f9 pod in namespace openshift-image-registry (previous: false). I0505 23:19:14.482832 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-746546b4d6-lx4f9 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-746546b4d6-lx4f9\" is waiting to start: ContainerCreating" I0505 23:19:14.482850 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-746546b4d6-lx4f9\" is waiting to start: ContainerCreating" I0505 23:19:14.482882 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-c9f9bbd8f-mgpvj pod in namespace openshift-image-registry (previous: false). I0505 23:19:14.684892 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-c9f9bbd8f-mgpvj pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-c9f9bbd8f-mgpvj\" is waiting to start: ContainerCreating" I0505 23:19:14.684914 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-c9f9bbd8f-mgpvj\" is waiting to start: ContainerCreating" I0505 23:19:14.684927 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-bsmn2 pod in namespace openshift-image-registry (previous: false). I0505 23:19:14.885625 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 23:19:14.885645 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-psz4j pod in namespace openshift-image-registry (previous: false). W0505 23:19:15.071807 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 23:19:15.088728 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 23:19:15.088753 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-w9vr8 pod in namespace openshift-image-registry (previous: false). I0505 23:19:15.288072 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 23:19:15.288095 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7ff9b5cb8c-nfp5w pod in namespace openshift-ingress (previous: false). I0505 23:19:15.482898 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7ff9b5cb8c-nfp5w pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7ff9b5cb8c-nfp5w\" is waiting to start: ContainerCreating" I0505 23:19:15.482919 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7ff9b5cb8c-nfp5w\" is waiting to start: ContainerCreating" I0505 23:19:15.482931 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-d9c9f5966-75lqw pod in namespace openshift-ingress (previous: false). I0505 23:19:15.683814 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-d9c9f5966-75lqw pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-d9c9f5966-75lqw\" is waiting to start: ContainerCreating" I0505 23:19:15.683833 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-d9c9f5966-75lqw\" is waiting to start: ContainerCreating" I0505 23:19:15.683845 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-d9c9f5966-jf4j9 pod in namespace openshift-ingress (previous: false). I0505 23:19:15.883101 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-d9c9f5966-jf4j9 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-d9c9f5966-jf4j9\" is waiting to start: ContainerCreating" I0505 23:19:15.883123 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-d9c9f5966-jf4j9\" is waiting to start: ContainerCreating" I0505 23:19:15.883156 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-t48jc pod in namespace openshift-ingress-canary (previous: false). W0505 23:19:16.070607 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 23:19:16.070637 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0505 23:19:16.070655 1 tasks_processing.go:74] worker 18 stopped. E0505 23:19:16.070666 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0505 23:19:16.070678 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0505 23:19:16.070691 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0505 23:19:16.070699 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.049256967s to process 1 records I0505 23:19:16.083207 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-t48jc pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t48jc\" is waiting to start: ContainerCreating" I0505 23:19:16.083231 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t48jc\" is waiting to start: ContainerCreating" I0505 23:19:16.083275 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-v7rhz pod in namespace openshift-ingress-canary (previous: false). I0505 23:19:16.283698 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-v7rhz pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-v7rhz\" is waiting to start: ContainerCreating" I0505 23:19:16.283717 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-v7rhz\" is waiting to start: ContainerCreating" I0505 23:19:16.283744 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-vhwhk pod in namespace openshift-ingress-canary (previous: false). I0505 23:19:16.486799 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-vhwhk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-vhwhk\" is waiting to start: ContainerCreating" I0505 23:19:16.486820 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-vhwhk\" is waiting to start: ContainerCreating" I0505 23:19:16.486834 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-8684n pod in namespace openshift-kube-storage-version-migrator (previous: false). I0505 23:19:16.696358 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-8684n pod in namespace openshift-kube-storage-version-migrator (previous: false). I0505 23:19:16.888130 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-qcftj pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0505 23:19:17.085018 1 tasks_processing.go:74] worker 15 stopped. I0505 23:19:17.085132 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=1d702bcd9f2e9f1a4af02e21153cab863eddef1c134c4de17c2dfd26d6d9ad0f I0505 23:19:17.085197 1 recorder.go:75] Recording events/openshift-dns with fingerprint=51ff940c266f09da05356d2358995b7791440770a5180792ea5d2b1709988d9f I0505 23:19:17.085314 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=b3486cc329415ee1470f0b4bd64eee1cd665338ae3d5145b77844956c533c059 I0505 23:19:17.085354 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=5f1c3f6ef12cd74c556f73bb4828212244a26105a9b22eac08d386f4c32dc562 I0505 23:19:17.085435 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=ea645643d6e520977aca86557f37ccffb83c777692ff3e434b33e2cdc4a2e333 I0505 23:19:17.085464 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=06004f697329800353ebe083a3269f543fad30a8b47a96efa250f413d2b4ddc5 I0505 23:19:17.085490 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=8921506f7151ad357be3e6963ef93136a30928587a608555f71c63ce54ad1680 I0505 23:19:17.085556 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=a84d7bbacd62b7f0749852cccf9bcc9023bbd9a0b38d3664b41123a07bfea5c8 I0505 23:19:17.085706 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-9mlvg with fingerprint=b8132a5d1e4632b405103b551db7a5ee844a23653d4349a8b95d8fb4b9b2223b I0505 23:19:17.085801 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-bpb4w with fingerprint=9dca5e6f42c6ebbb0935999d15c0ea6b134e7a50239adca44ae0bed5b689ce5a I0505 23:19:17.085888 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-t6tb8 with fingerprint=0ec1113fc717514af6c424eeab6307b77395320e8228cf8e0ef7019beea09cef I0505 23:19:17.086031 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-746546b4d6-fsxkc with fingerprint=d2ca8acc2b1b4d61be92af081d31dbcd07e04cf0e8a12797d4d12e7ae57505fc I0505 23:19:17.086143 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-746546b4d6-lx4f9 with fingerprint=a994dd0905ca020fb39fd16bee4ff12c88e199c5a7cc58d4a6edf304e924e225 I0505 23:19:17.086246 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-c9f9bbd8f-mgpvj with fingerprint=d1c4c5eedc5f2c79fc357f0f3865fea94f82c6eadd023edfc0ea7eb62b9efd5f I0505 23:19:17.086321 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-t48jc with fingerprint=5113072abccbdaf82cd6b5a03bde992f196c6e5ddd3cf569d197b20314764a47 I0505 23:19:17.086389 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-v7rhz with fingerprint=8901d15bbedcfbe80d83567895a69f5d76fd2275118737a337ec91f7fcf539ba I0505 23:19:17.086454 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-vhwhk with fingerprint=e2d134d3fda60f357ea27ea5bbdd5ab5011419cfc5a4f5ceb80112cddd1fd8c9 I0505 23:19:17.086480 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-8684n/migrator_current.log with fingerprint=24dcf8ec160b8a8b31189360c7be3d2990dc867b11c92574a3552f9ee9e1923f I0505 23:19:17.086490 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-8684n/graceful-termination_current.log with fingerprint=3c0aa6aa5ffe1987d5024b02c37dea5352297ebe8d18ebd824114221650da74e I0505 23:19:17.086577 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-qcftj/kube-storage-version-migrator-operator_current.log with fingerprint=9a57809428acfcb572a2cd10ddfe4f6c8a3ab00116eeea003597533e97b0c59c I0505 23:19:17.086590 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.06299655s to process 20 records I0505 23:19:23.476851 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 23:19:23.680144 1 tasks_processing.go:74] worker 11 stopped. I0505 23:19:23.680197 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0505 23:19:23.680213 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.658514046s to process 1 records I0505 23:19:24.043186 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 23:19:24.235249 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 23:19:24.430116 1 tasks_processing.go:74] worker 63 stopped. I0505 23:19:24.430405 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=0088e0a32371311f7e1870b984c632b0a0bea653dc12cdf0ea7947cd6c4546f3 I0505 23:19:24.430428 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.411139168s to process 1 records E0505 23:19:24.430497 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.411s with: function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0505 23:19:24.431609 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0505 23:19:24.431626 1 periodic.go:209] Running workloads gatherer I0505 23:19:24.431641 1 tasks_processing.go:45] number of workers: 2 I0505 23:19:24.431647 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 23:19:24.431651 1 tasks_processing.go:71] worker 1 working on workload_info task. I0505 23:19:24.431714 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 23:19:24.431784 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0505 23:19:24.460739 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0505 23:19:24.471129 1 tasks_processing.go:74] worker 0 stopped. I0505 23:19:24.471147 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 39.32389ms to process 0 records I0505 23:19:24.472448 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (12ms) I0505 23:19:24.483140 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (11ms) I0505 23:19:24.494426 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (11ms) I0505 23:19:24.505361 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (11ms) I0505 23:19:24.516832 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (11ms) I0505 23:19:24.527624 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (11ms) I0505 23:19:24.538674 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (11ms) I0505 23:19:24.549922 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (11ms) I0505 23:19:24.562499 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (13ms) I0505 23:19:24.573445 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (11ms) I0505 23:19:24.584106 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (11ms) I0505 23:19:24.671508 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (87ms) I0505 23:19:24.772551 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (101ms) I0505 23:19:24.871710 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0505 23:19:24.971976 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0505 23:19:25.072008 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0505 23:19:25.172090 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0505 23:19:25.271368 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0505 23:19:25.372093 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0505 23:19:25.471336 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (99ms) I0505 23:19:25.571913 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (101ms) I0505 23:19:25.681484 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (110ms) I0505 23:19:25.771918 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (90ms) I0505 23:19:25.871143 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0505 23:19:25.971357 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0505 23:19:26.072298 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (101ms) I0505 23:19:26.171806 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (99ms) I0505 23:19:26.272696 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0505 23:19:26.372390 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (100ms) I0505 23:19:26.472170 1 gather_workloads_info.go:387] No image sha256:f68c0da8268f23078f5f760a91a3c8212f2195acf18c231f6e034207a67adeec (100ms) I0505 23:19:26.571350 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (99ms) I0505 23:19:26.671875 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0505 23:19:26.771074 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0505 23:19:26.771114 1 tasks_processing.go:74] worker 1 stopped. E0505 23:19:26.771127 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0505 23:19:26.771493 1 recorder.go:75] Recording config/workload_info with fingerprint=4c3758b02d699717e55441b7184413763828be147b8b0ebaff4fca236401ddac I0505 23:19:26.771536 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.339455145s to process 1 records E0505 23:19:26.771572 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.339s with: function \"workload_info\" failed with an error" I0505 23:19:26.772683 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0505 23:19:26.772697 1 periodic.go:209] Running conditional gatherer I0505 23:19:26.780538 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0505 23:19:26.786975 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.15:34082->172.30.0.10:53: read: connection refused E0505 23:19:26.787244 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0505 23:19:26.787315 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0505 23:19:26.795442 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0505 23:19:26.795465 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795472 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795477 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795482 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795486 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795490 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795494 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795498 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0505 23:19:26.795503 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0505 23:19:26.795525 1 tasks_processing.go:45] number of workers: 3 I0505 23:19:26.795541 1 tasks_processing.go:69] worker 2 listening for tasks. I0505 23:19:26.795548 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0505 23:19:26.795552 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 23:19:26.795559 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 23:19:26.795567 1 tasks_processing.go:74] worker 1 stopped. I0505 23:19:26.795568 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0505 23:19:26.795582 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0505 23:19:26.795661 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0505 23:19:26.795697 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.587µs to process 1 records I0505 23:19:26.795744 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0505 23:19:26.795756 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 2.025µs to process 1 records I0505 23:19:26.795765 1 tasks_processing.go:74] worker 0 stopped. I0505 23:19:26.795856 1 tasks_processing.go:74] worker 2 stopped. I0505 23:19:26.795869 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 257.61µs to process 0 records I0505 23:19:26.795904 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.15:34082->172.30.0.10:53: read: connection refused I0505 23:19:26.795924 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0505 23:19:26.822471 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=846bc618b56660b8c32a2b20ed4ded9846f667b066634cb9f16481c09e55972f I0505 23:19:26.822615 1 diskrecorder.go:70] Writing 116 records to /var/lib/insights-operator/insights-2026-05-05-231926.tar.gz I0505 23:19:26.830403 1 diskrecorder.go:51] Wrote 116 records to disk in 7ms I0505 23:19:26.830461 1 periodic.go:278] Gathering cluster info every 2h0m0s I0505 23:19:26.830480 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0505 23:19:32.165988 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 23:20:39.995971 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="cce70c59e0a3a73cbc39de245b185d509313b87e7fda3889c7f04078ffbb747c") W0505 23:20:39.996009 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0505 23:20:39.996056 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="bf702971e54e0af855a834c8b0c751a0184b31429abd599d55c2027d5c015551") I0505 23:20:39.996072 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0505 23:20:39.996097 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0505 23:20:39.996101 1 periodic.go:170] Shutting down I0505 23:20:39.996133 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="9132f42b8fd36bbe5e58f380ca7f84d655cee1f9fc8f58f6f64f4bddfea5d906") I0505 23:20:39.996141 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0505 23:20:39.996157 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain"