W0428 15:53:01.763871 1 cmd.go:257] Using insecure, self-signed certificates I0428 15:53:02.632245 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0428 15:53:02.632566 1 observer_polling.go:159] Starting file observer I0428 15:53:03.367677 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0428 15:53:03.367862 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0428 15:53:03.368409 1 secure_serving.go:57] Forcing use of http/1.1 only W0428 15:53:03.368428 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0428 15:53:03.368432 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0428 15:53:03.368435 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. I0428 15:53:03.368434 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0428 15:53:03.368438 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0428 15:53:03.368454 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0428 15:53:03.368456 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0428 15:53:03.371571 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0428 15:53:03.371601 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"09068850-fd87-4d2c-a103-d55569c80a1d", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0428 15:53:03.371858 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0428 15:53:03.371871 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0428 15:53:03.371917 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0428 15:53:03.371939 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0428 15:53:03.371918 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0428 15:53:03.371952 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0428 15:53:03.372523 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1159620006/tls.crt::/tmp/serving-cert-1159620006/tls.key" I0428 15:53:03.372973 1 secure_serving.go:213] Serving securely on [::]:8443 I0428 15:53:03.373017 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0428 15:53:03.378963 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0428 15:53:03.378988 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0428 15:53:03.379018 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0428 15:53:03.388501 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0428 15:53:03.388521 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0428 15:53:03.391940 1 secretconfigobserver.go:119] support secret does not exist I0428 15:53:03.396736 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0428 15:53:03.401599 1 secretconfigobserver.go:119] support secret does not exist I0428 15:53:03.403721 1 recorder.go:161] Pruning old reports every 7h21m1s, max age is 288h0m0s I0428 15:53:03.408440 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0428 15:53:03.408454 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0428 15:53:03.408461 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0428 15:53:03.408473 1 insightsreport.go:296] Starting report retriever I0428 15:53:03.408478 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0428 15:53:03.408490 1 periodic.go:209] Running clusterconfig gatherer I0428 15:53:03.408531 1 tasks_processing.go:45] number of workers: 64 I0428 15:53:03.408556 1 tasks_processing.go:69] worker 2 listening for tasks. I0428 15:53:03.408565 1 tasks_processing.go:69] worker 1 listening for tasks. I0428 15:53:03.408565 1 tasks_processing.go:69] worker 0 listening for tasks. I0428 15:53:03.408571 1 tasks_processing.go:69] worker 27 listening for tasks. I0428 15:53:03.408573 1 tasks_processing.go:69] worker 14 listening for tasks. I0428 15:53:03.408575 1 tasks_processing.go:71] worker 0 working on machine_autoscalers task. I0428 15:53:03.408579 1 tasks_processing.go:69] worker 9 listening for tasks. I0428 15:53:03.408579 1 tasks_processing.go:71] worker 14 working on sap_config task. I0428 15:53:03.408583 1 tasks_processing.go:71] worker 9 working on sap_pods task. I0428 15:53:03.408594 1 tasks_processing.go:69] worker 20 listening for tasks. I0428 15:53:03.408595 1 tasks_processing.go:69] worker 8 listening for tasks. I0428 15:53:03.408604 1 tasks_processing.go:69] worker 15 listening for tasks. I0428 15:53:03.408608 1 tasks_processing.go:69] worker 16 listening for tasks. I0428 15:53:03.408614 1 tasks_processing.go:69] worker 17 listening for tasks. I0428 15:53:03.408618 1 tasks_processing.go:69] worker 46 listening for tasks. I0428 15:53:03.408622 1 tasks_processing.go:69] worker 18 listening for tasks. I0428 15:53:03.408624 1 tasks_processing.go:69] worker 3 listening for tasks. I0428 15:53:03.408629 1 tasks_processing.go:69] worker 28 listening for tasks. I0428 15:53:03.408631 1 tasks_processing.go:69] worker 19 listening for tasks. I0428 15:53:03.408632 1 tasks_processing.go:69] worker 23 listening for tasks. I0428 15:53:03.408636 1 tasks_processing.go:69] worker 29 listening for tasks. I0428 15:53:03.408640 1 tasks_processing.go:69] worker 56 listening for tasks. I0428 15:53:03.408635 1 tasks_processing.go:69] worker 4 listening for tasks. I0428 15:53:03.408648 1 tasks_processing.go:71] worker 56 working on storage_cluster task. I0428 15:53:03.408649 1 tasks_processing.go:69] worker 22 listening for tasks. I0428 15:53:03.408649 1 tasks_processing.go:69] worker 31 listening for tasks. I0428 15:53:03.408653 1 tasks_processing.go:69] worker 6 listening for tasks. I0428 15:53:03.408656 1 tasks_processing.go:71] worker 22 working on machine_configs task. I0428 15:53:03.408658 1 tasks_processing.go:69] worker 39 listening for tasks. I0428 15:53:03.408657 1 tasks_processing.go:69] worker 61 listening for tasks. I0428 15:53:03.408663 1 tasks_processing.go:69] worker 7 listening for tasks. I0428 15:53:03.408658 1 tasks_processing.go:69] worker 47 listening for tasks. I0428 15:53:03.408651 1 tasks_processing.go:71] worker 4 working on networks task. I0428 15:53:03.408672 1 tasks_processing.go:69] worker 57 listening for tasks. I0428 15:53:03.408667 1 tasks_processing.go:69] worker 41 listening for tasks. I0428 15:53:03.408679 1 tasks_processing.go:69] worker 49 listening for tasks. I0428 15:53:03.408682 1 tasks_processing.go:69] worker 44 listening for tasks. I0428 15:53:03.408671 1 tasks_processing.go:69] worker 48 listening for tasks. I0428 15:53:03.408688 1 tasks_processing.go:69] worker 58 listening for tasks. I0428 15:53:03.408689 1 tasks_processing.go:69] worker 50 listening for tasks. I0428 15:53:03.408673 1 tasks_processing.go:69] worker 42 listening for tasks. I0428 15:53:03.408693 1 tasks_processing.go:69] worker 45 listening for tasks. I0428 15:53:03.408680 1 tasks_processing.go:69] worker 38 listening for tasks. I0428 15:53:03.408694 1 tasks_processing.go:69] worker 63 listening for tasks. I0428 15:53:03.408694 1 tasks_processing.go:69] worker 32 listening for tasks. I0428 15:53:03.408678 1 tasks_processing.go:69] worker 43 listening for tasks. I0428 15:53:03.408699 1 tasks_processing.go:69] worker 51 listening for tasks. I0428 15:53:03.408708 1 tasks_processing.go:69] worker 37 listening for tasks. I0428 15:53:03.408713 1 tasks_processing.go:69] worker 54 listening for tasks. I0428 15:53:03.408720 1 tasks_processing.go:69] worker 53 listening for tasks. I0428 15:53:03.408645 1 tasks_processing.go:69] worker 30 listening for tasks. I0428 15:53:03.408722 1 tasks_processing.go:71] worker 2 working on silenced_alerts task. I0428 15:53:03.408726 1 tasks_processing.go:69] worker 34 listening for tasks. I0428 15:53:03.408728 1 tasks_processing.go:69] worker 26 listening for tasks. I0428 15:53:03.408729 1 tasks_processing.go:69] worker 24 listening for tasks. I0428 15:53:03.408731 1 tasks_processing.go:69] worker 10 listening for tasks. I0428 15:53:03.408705 1 tasks_processing.go:69] worker 62 listening for tasks. I0428 15:53:03.408739 1 tasks_processing.go:71] worker 20 working on machine_healthchecks task. I0428 15:53:03.408742 1 tasks_processing.go:71] worker 62 working on sap_datahubs task. I0428 15:53:03.408743 1 tasks_processing.go:71] worker 46 working on openstack_controlplanes task. I0428 15:53:03.408745 1 tasks_processing.go:69] worker 12 listening for tasks. I0428 15:53:03.408737 1 tasks_processing.go:69] worker 33 listening for tasks. I0428 15:53:03.408748 1 tasks_processing.go:71] worker 31 working on authentication task. I0428 15:53:03.408644 1 tasks_processing.go:69] worker 21 listening for tasks. I0428 15:53:03.408713 1 tasks_processing.go:69] worker 60 listening for tasks. W0428 15:53:03.408756 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0428 15:53:03.408700 1 tasks_processing.go:69] worker 59 listening for tasks. I0428 15:53:03.408756 1 tasks_processing.go:69] worker 11 listening for tasks. I0428 15:53:03.408769 1 tasks_processing.go:71] worker 42 working on qemu_kubevirt_launcher_logs task. I0428 15:53:03.408771 1 tasks_processing.go:71] worker 28 working on operators task. I0428 15:53:03.408772 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 35.106µs to process 0 records I0428 15:53:03.408575 1 tasks_processing.go:71] worker 27 working on monitoring_persistent_volumes task. I0428 15:53:03.408782 1 tasks_processing.go:71] worker 23 working on ceph_cluster task. I0428 15:53:03.408780 1 tasks_processing.go:71] worker 18 working on config_maps task. I0428 15:53:03.408805 1 tasks_processing.go:71] worker 11 working on storage_classes task. I0428 15:53:03.408820 1 tasks_processing.go:71] worker 29 working on install_plans task. I0428 15:53:03.408836 1 tasks_processing.go:71] worker 6 working on olm_operators task. I0428 15:53:03.408841 1 tasks_processing.go:71] worker 54 working on pod_network_connectivity_checks task. I0428 15:53:03.408701 1 tasks_processing.go:69] worker 36 listening for tasks. I0428 15:53:03.408738 1 tasks_processing.go:71] worker 17 working on nodes task. I0428 15:53:03.409082 1 tasks_processing.go:71] worker 45 working on pdbs task. I0428 15:53:03.409086 1 tasks_processing.go:71] worker 15 working on overlapping_namespace_uids task. I0428 15:53:03.409094 1 tasks_processing.go:71] worker 53 working on mutating_webhook_configurations task. I0428 15:53:03.409142 1 tasks_processing.go:71] worker 36 working on machine_config_pools task. I0428 15:53:03.409226 1 tasks_processing.go:71] worker 34 working on oauths task. I0428 15:53:03.409250 1 tasks_processing.go:71] worker 38 working on infrastructures task. I0428 15:53:03.409259 1 tasks_processing.go:71] worker 63 working on operators_pods_and_events task. I0428 15:53:03.409327 1 tasks_processing.go:71] worker 41 working on openstack_dataplanedeployments task. I0428 15:53:03.408706 1 tasks_processing.go:69] worker 52 listening for tasks. I0428 15:53:03.408734 1 tasks_processing.go:71] worker 3 working on openstack_dataplanenodesets task. I0428 15:53:03.409448 1 tasks_processing.go:71] worker 48 working on cost_management_metrics_configs task. I0428 15:53:03.409552 1 tasks_processing.go:71] worker 43 working on image_registries task. I0428 15:53:03.409559 1 tasks_processing.go:71] worker 30 working on cluster_apiserver task. I0428 15:53:03.408737 1 tasks_processing.go:69] worker 13 listening for tasks. I0428 15:53:03.408743 1 tasks_processing.go:71] worker 10 working on schedulers task. I0428 15:53:03.409668 1 tasks_processing.go:71] worker 24 working on image task. I0428 15:53:03.409723 1 tasks_processing.go:71] worker 50 working on tsdb_status task. I0428 15:53:03.408717 1 tasks_processing.go:69] worker 35 listening for tasks. I0428 15:53:03.408761 1 tasks_processing.go:71] worker 1 working on clusterroles task. I0428 15:53:03.408767 1 tasks_processing.go:71] worker 19 working on jaegers task. I0428 15:53:03.408714 1 tasks_processing.go:69] worker 55 listening for tasks. I0428 15:53:03.408722 1 tasks_processing.go:69] worker 25 listening for tasks. I0428 15:53:03.408726 1 tasks_processing.go:71] worker 16 working on container_runtime_configs task. I0428 15:53:03.409606 1 tasks_processing.go:71] worker 26 working on dvo_metrics task. I0428 15:53:03.409809 1 tasks_processing.go:71] worker 61 working on machine_sets task. I0428 15:53:03.409829 1 tasks_processing.go:71] worker 58 working on nodenetworkstates task. I0428 15:53:03.409908 1 tasks_processing.go:71] worker 51 working on image_pruners task. I0428 15:53:03.408754 1 tasks_processing.go:71] worker 8 working on version task. I0428 15:53:03.410058 1 tasks_processing.go:71] worker 7 working on service_accounts task. I0428 15:53:03.409250 1 tasks_processing.go:71] worker 32 working on aggregated_monitoring_cr_names task. I0428 15:53:03.410167 1 tasks_processing.go:71] worker 49 working on nodenetworkconfigurationpolicies task. I0428 15:53:03.410205 1 tasks_processing.go:71] worker 47 working on support_secret task. I0428 15:53:03.410277 1 tasks_processing.go:71] worker 13 working on openshift_logging task. I0428 15:53:03.410300 1 tasks_processing.go:71] worker 44 working on active_alerts task. W0428 15:53:03.410332 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0428 15:53:03.408643 1 tasks_processing.go:69] worker 5 listening for tasks. I0428 15:53:03.410436 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 70.712µs to process 0 records I0428 15:53:03.410458 1 tasks_processing.go:71] worker 57 working on container_images task. I0428 15:53:03.410465 1 tasks_processing.go:71] worker 35 working on openstack_version task. I0428 15:53:03.408663 1 tasks_processing.go:69] worker 40 listening for tasks. I0428 15:53:03.408838 1 tasks_processing.go:71] worker 37 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0428 15:53:03.410674 1 tasks_processing.go:74] worker 40 stopped. I0428 15:53:03.410697 1 tasks_processing.go:71] worker 33 working on machines task. I0428 15:53:03.410760 1 tasks_processing.go:71] worker 2 working on proxies task. I0428 15:53:03.409730 1 tasks_processing.go:71] worker 39 working on ingress task. I0428 15:53:03.410697 1 tasks_processing.go:71] worker 21 working on validating_webhook_configurations task. I0428 15:53:03.410823 1 tasks_processing.go:71] worker 12 working on crds task. I0428 15:53:03.410911 1 tasks_processing.go:71] worker 44 working on lokistack task. W0428 15:53:03.410368 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0428 15:53:03.410963 1 tasks_processing.go:74] worker 50 stopped. I0428 15:53:03.410976 1 tasks_processing.go:71] worker 25 working on node_logs task. I0428 15:53:03.410987 1 tasks_processing.go:71] worker 5 working on ingress_certificates task. I0428 15:53:03.410990 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 605.547µs to process 0 records I0428 15:53:03.411469 1 tasks_processing.go:71] worker 59 working on feature_gates task. I0428 15:53:03.411955 1 tasks_processing.go:71] worker 60 working on metrics task. W0428 15:53:03.411981 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0428 15:53:03.411992 1 tasks_processing.go:74] worker 60 stopped. I0428 15:53:03.411999 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 25.865µs to process 0 records I0428 15:53:03.412262 1 tasks_processing.go:71] worker 52 working on certificate_signing_requests task. I0428 15:53:03.412495 1 tasks_processing.go:74] worker 0 stopped. I0428 15:53:03.412508 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 3.912383ms to process 0 records I0428 15:53:03.413012 1 tasks_processing.go:71] worker 55 working on openshift_machine_api_events task. I0428 15:53:03.415761 1 tasks_processing.go:74] worker 14 stopped. I0428 15:53:03.415778 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 7.172127ms to process 0 records E0428 15:53:03.415790 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0428 15:53:03.415796 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 6.777144ms to process 0 records I0428 15:53:03.415801 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 7.019515ms to process 0 records I0428 15:53:03.415805 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 6.990225ms to process 0 records I0428 15:53:03.415808 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 7.14494ms to process 0 records I0428 15:53:03.415814 1 tasks_processing.go:74] worker 56 stopped. I0428 15:53:03.415817 1 tasks_processing.go:74] worker 54 stopped. I0428 15:53:03.415820 1 tasks_processing.go:74] worker 62 stopped. I0428 15:53:03.415823 1 tasks_processing.go:74] worker 23 stopped. I0428 15:53:03.415994 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0428 15:53:03.416009 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0428 15:53:03.416014 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0428 15:53:03.416017 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0428 15:53:03.416030 1 controller.go:489] The operator is still being initialized I0428 15:53:03.416038 1 controller.go:512] The operator is healthy I0428 15:53:03.426936 1 tasks_processing.go:74] worker 4 stopped. I0428 15:53:03.427100 1 recorder.go:75] Recording config/network with fingerprint=34c7b8eb91fabfb6e311b111afb5637b3f7f49b4dccec35c1eff74da94a7b96e I0428 15:53:03.427113 1 gather.go:177] gatherer "clusterconfig" function "networks" took 18.252002ms to process 1 records I0428 15:53:03.448987 1 tasks_processing.go:74] worker 61 stopped. I0428 15:53:03.449001 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 39.160543ms to process 0 records I0428 15:53:03.449009 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 39.150278ms to process 0 records I0428 15:53:03.449014 1 tasks_processing.go:74] worker 58 stopped. I0428 15:53:03.449293 1 tasks_processing.go:74] worker 41 stopped. I0428 15:53:03.449306 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 39.944043ms to process 0 records E0428 15:53:03.449317 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0428 15:53:03.449323 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 39.096409ms to process 0 records I0428 15:53:03.449329 1 tasks_processing.go:74] worker 47 stopped. I0428 15:53:03.454417 1 tasks_processing.go:74] worker 49 stopped. I0428 15:53:03.454432 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 44.23706ms to process 0 records I0428 15:53:03.454442 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 44.14333ms to process 0 records I0428 15:53:03.454446 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 43.961465ms to process 0 records I0428 15:53:03.454451 1 tasks_processing.go:74] worker 35 stopped. I0428 15:53:03.454452 1 tasks_processing.go:74] worker 13 stopped. I0428 15:53:03.454629 1 tasks_processing.go:74] worker 46 stopped. I0428 15:53:03.454642 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 45.878866ms to process 0 records I0428 15:53:03.454735 1 tasks_processing.go:74] worker 48 stopped. I0428 15:53:03.454743 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 45.278732ms to process 0 records I0428 15:53:03.454765 1 tasks_processing.go:74] worker 45 stopped. I0428 15:53:03.454998 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=3e1adebc46e944aabe712078102525c99d0de134787a3d39e75145773011191c I0428 15:53:03.455030 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=86be24a569d5b40ce2eb5cec9955db44442f12755d33c09191555ff994d0fa72 I0428 15:53:03.455054 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=231c72f4d2aa93694fc4054572c975d16d77ca7b0b6414de33ed2f786294987f I0428 15:53:03.455068 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 45.666792ms to process 3 records I0428 15:53:03.455205 1 tasks_processing.go:74] worker 34 stopped. I0428 15:53:03.455371 1 recorder.go:75] Recording config/oauth with fingerprint=efe3d7b19a47004f477f6a972e4fdf026be060866597877636bad372d6b9ea3a I0428 15:53:03.455387 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 45.554113ms to process 1 records I0428 15:53:03.455394 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 44.427283ms to process 0 records I0428 15:53:03.455401 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 46.199991ms to process 0 records I0428 15:53:03.455430 1 tasks_processing.go:74] worker 16 stopped. I0428 15:53:03.455442 1 tasks_processing.go:74] worker 27 stopped. I0428 15:53:03.455520 1 recorder.go:75] Recording config/ingress with fingerprint=c7527b0c82eef85a217f775e1bafd913e195cfa641b627f63cb43beb709957e4 I0428 15:53:03.455530 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 44.200896ms to process 1 records I0428 15:53:03.455533 1 tasks_processing.go:74] worker 39 stopped. E0428 15:53:03.455536 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0428 15:53:03.455543 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 46.288584ms to process 0 records I0428 15:53:03.455555 1 tasks_processing.go:74] worker 20 stopped. I0428 15:53:03.455613 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=b2b05e558dbabf4ab8d5723d7cfc7cc1beb3a98cf7773f56a0d5aeaa84a56a7c I0428 15:53:03.455630 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=09427f55751b795dbd91e549ead34a70140461732a6ed0291cc441008cfc302e I0428 15:53:03.455635 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 46.273642ms to process 2 records I0428 15:53:03.455635 1 tasks_processing.go:74] worker 11 stopped. E0428 15:53:03.455640 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0428 15:53:03.455646 1 gather.go:177] gatherer "clusterconfig" function "machines" took 44.338595ms to process 0 records I0428 15:53:03.455650 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 42.092402ms to process 0 records I0428 15:53:03.455659 1 tasks_processing.go:74] worker 33 stopped. I0428 15:53:03.455668 1 tasks_processing.go:74] worker 55 stopped. I0428 15:53:03.455747 1 tasks_processing.go:74] worker 38 stopped. I0428 15:53:03.456331 1 recorder.go:75] Recording config/infrastructure with fingerprint=fdc12ca626231594e1d29babdd34617554e86ced6b118e8ce72c0426f2c2f1aa I0428 15:53:03.456348 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 46.095679ms to process 1 records I0428 15:53:03.456467 1 tasks_processing.go:74] worker 30 stopped. I0428 15:53:03.456505 1 recorder.go:75] Recording config/apiserver with fingerprint=02108e5ef838ecc70bfabcab40796c2ca6e78ec392653f06330ec33d9133962d I0428 15:53:03.456515 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 45.876815ms to process 1 records I0428 15:53:03.456563 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=4e69fa6f8626433afbef1ac49c11a523887fcf21c563d8ef16ae9a1d4dfa4611 I0428 15:53:03.456571 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 45.848204ms to process 1 records I0428 15:53:03.456576 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 47.04926ms to process 0 records I0428 15:53:03.456581 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 44.727965ms to process 0 records I0428 15:53:03.456586 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 46.807513ms to process 0 records I0428 15:53:03.456596 1 tasks_processing.go:74] worker 9 stopped. I0428 15:53:03.456596 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 45.348256ms to process 0 records I0428 15:53:03.456610 1 tasks_processing.go:74] worker 3 stopped. I0428 15:53:03.456605 1 tasks_processing.go:74] worker 25 stopped. I0428 15:53:03.456613 1 tasks_processing.go:74] worker 44 stopped. I0428 15:53:03.456618 1 tasks_processing.go:74] worker 10 stopped. I0428 15:53:03.456684 1 tasks_processing.go:74] worker 24 stopped. I0428 15:53:03.456703 1 recorder.go:75] Recording config/image with fingerprint=cbc7ce067397b413123056f346c45d3ec07fc40e2ea1451c785ba4eabd9d9426 I0428 15:53:03.456712 1 gather.go:177] gatherer "clusterconfig" function "image" took 46.794964ms to process 1 records I0428 15:53:03.456783 1 tasks_processing.go:74] worker 51 stopped. I0428 15:53:03.456889 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=9f14e63f76cd717a16c85d890698e9ed4e4611a2b88613766671bd1e3ebf195a I0428 15:53:03.456900 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 46.741626ms to process 1 records I0428 15:53:03.458270 1 tasks_processing.go:74] worker 2 stopped. I0428 15:53:03.458466 1 recorder.go:75] Recording config/proxy with fingerprint=00ec52a8b5af2ace6002076071794b6604ff3979b5352ce67828defd15f2587d I0428 15:53:03.458532 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 47.49517ms to process 1 records I0428 15:53:03.458750 1 tasks_processing.go:74] worker 19 stopped. I0428 15:53:03.458810 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 48.714807ms to process 0 records I0428 15:53:03.458905 1 gather_logs.go:145] no pods in namespace were found I0428 15:53:03.458964 1 tasks_processing.go:74] worker 42 stopped. I0428 15:53:03.458996 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 50.186343ms to process 0 records I0428 15:53:03.459111 1 tasks_processing.go:74] worker 59 stopped. I0428 15:53:03.459443 1 recorder.go:75] Recording config/featuregate with fingerprint=7342d6e6f32e68f98f55aecde72c0abcb854d45f3e918e2a272946abca8e288c I0428 15:53:03.459518 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 47.624595ms to process 1 records I0428 15:53:03.459574 1 tasks_processing.go:74] worker 17 stopped. I0428 15:53:03.460146 1 recorder.go:75] Recording config/node/ip-10-0-0-43.ec2.internal with fingerprint=be709b8b9b3dc533730fc8b138f7b5a5e989a5b3a067ef990432c28b9e37576a I0428 15:53:03.460292 1 recorder.go:75] Recording config/node/ip-10-0-1-151.ec2.internal with fingerprint=1c5bd98cb3b446aa851c6a811d517d97717972a7dd56bb55aa0572746ccc8cbe I0428 15:53:03.460410 1 recorder.go:75] Recording config/node/ip-10-0-2-223.ec2.internal with fingerprint=7ed5332077e93fdbdd1c1e7ef572fc5b876a616f87c9218f1235b9bdeb0d314f I0428 15:53:03.460421 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 50.503944ms to process 3 records I0428 15:53:03.460468 1 tasks_processing.go:74] worker 31 stopped. I0428 15:53:03.462063 1 recorder.go:75] Recording config/authentication with fingerprint=7f75fd2327c692ce983f0a7b97b0aedfaf27e286169a378c2bc7c1a2b2fa22e3 I0428 15:53:03.462087 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 51.548101ms to process 1 records I0428 15:53:03.462268 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=010801c21c87fdbb306411fb60b76107e5747bb790b67067a03d0accb2b1d759 I0428 15:53:03.462305 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=18b6698c24f4b463858fdd68f1c8446b83460816b63c623d8bcca48b15c42d04 I0428 15:53:03.462331 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=e399cc438a52d29229a4f99c0e00f13b33aa493543c58338259d33ec419ae51c I0428 15:53:03.462341 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 52.928793ms to process 3 records I0428 15:53:03.462352 1 tasks_processing.go:74] worker 53 stopped. I0428 15:53:03.462533 1 tasks_processing.go:74] worker 43 stopped. I0428 15:53:03.462679 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=6f89773a59a761e1fdda2d669bb77462dfd6ee4a6b87d0ba822160bd650f6644 I0428 15:53:03.462689 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 52.576265ms to process 1 records I0428 15:53:03.462790 1 tasks_processing.go:74] worker 21 stopped. I0428 15:53:03.462802 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=80562c0ceb70e18699a7e0f038b376edd40e4d5f57fb8092acd8f4927e35b300 I0428 15:53:03.462885 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=4b793ee0e8020e798fa2544f86f42bad0c08e074ef3a51351396dbcf0e21b2a9 I0428 15:53:03.462912 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=a1cc2283525ef4fb0c7531d499c052ad36c19390c019f415bf956ded064c99b4 I0428 15:53:03.462937 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=3e6c17a7431fdca7a391afa03a62163d5becbc5b1f5e214d2fac966839692495 I0428 15:53:03.462962 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=8f55454c7353274b3051c53e9453ea5c97b5c41d0480df0a73a9c296e7e01eff I0428 15:53:03.462994 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=1bb665d32f35e496d60e4ab1d982f810e2bcc7bb5f680c1f5f72317011728735 I0428 15:53:03.463025 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=58574077903e8b11bb67424c30ff7260e3910e35b27c7e3b425e93e074d911ca I0428 15:53:03.463070 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=f0c7107b4b4cdb47e1cf66aede46b831db00b1b73734c98456491f6f6f20f649 I0428 15:53:03.463094 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=17bf89109b8e4330d34bd7c81c9a92d0f4510af0f23a6c0e7dcaa78744b528e6 I0428 15:53:03.463117 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=eb3ae508872b51b4da16e8efd4f3a360ca4f01cf49f81dd77d96f2842b3031f2 I0428 15:53:03.463149 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=6350819f7f4241d447ce646079aa5c2ed239b2f1a6eee60225eabb273bd16d3f I0428 15:53:03.463157 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 51.51814ms to process 11 records I0428 15:53:03.463168 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0428 15:53:03.463173 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 54.045028ms to process 1 records I0428 15:53:03.463180 1 tasks_processing.go:74] worker 15 stopped. I0428 15:53:03.463958 1 tasks_processing.go:74] worker 52 stopped. I0428 15:53:03.463972 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 51.677115ms to process 0 records W0428 15:53:03.465629 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0428 15:53:03.470945 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0428 15:53:03.471011 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0428 15:53:03.471127 1 operator.go:288] started I0428 15:53:03.471161 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0428 15:53:03.472785 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0428 15:53:03.472869 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0428 15:53:03.472883 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0428 15:53:03.479454 1 base_controller.go:82] Caches are synced for ConfigController I0428 15:53:03.479492 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0428 15:53:03.487334 1 tasks_processing.go:74] worker 6 stopped. I0428 15:53:03.487396 1 recorder.go:75] Recording config/olm_operators with fingerprint=bb661e1db9c9149c1e67806d6d75e896f845cddedd4d33326ed72ddf3644f2fa I0428 15:53:03.487410 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 78.479303ms to process 1 records I0428 15:53:03.488988 1 tasks_processing.go:74] worker 57 stopped. I0428 15:53:03.489082 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0428 15:53:03.489096 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0428 15:53:03.489102 1 controller.go:212] Source scaController *sca.Controller is not ready I0428 15:53:03.489108 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0428 15:53:03.489112 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0428 15:53:03.489138 1 controller.go:489] The operator is still being initialized I0428 15:53:03.489150 1 controller.go:512] The operator is healthy I0428 15:53:03.490179 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-1-151.ec2.internal with fingerprint=487f2bc3f7f220c8b85f956daa574c3da1c836599d88bf80f6e6ac18c2644f1d I0428 15:53:03.490243 1 recorder.go:75] Recording config/running_containers with fingerprint=221b51a208b4b37da505652906c685d9fc8ac25d9e97fded413d5d2fdf2dd0d9 I0428 15:53:03.490253 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 78.51174ms to process 2 records I0428 15:53:03.494230 1 tasks_processing.go:74] worker 8 stopped. I0428 15:53:03.495252 1 recorder.go:75] Recording config/version with fingerprint=5e57850fd033ab540855a082385e4a422b881621f57e1026da6445090ed71ac6 I0428 15:53:03.495328 1 recorder.go:75] Recording config/id with fingerprint=328650443d914bd5609d9e00822749af159500a224b1037a64c2fae06a791f50 I0428 15:53:03.495368 1 gather.go:177] gatherer "clusterconfig" function "version" took 84.295719ms to process 2 records I0428 15:53:03.495397 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 84.265959ms to process 0 records I0428 15:53:03.495427 1 tasks_processing.go:74] worker 37 stopped. I0428 15:53:03.495518 1 tasks_processing.go:74] worker 32 stopped. I0428 15:53:03.495596 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 85.345714ms to process 0 records I0428 15:53:03.497511 1 configmapobserver.go:84] configmaps "insights-config" not found I0428 15:53:03.497648 1 prometheus_rules.go:88] Prometheus rules successfully created I0428 15:53:03.498163 1 tasks_processing.go:74] worker 1 stopped. I0428 15:53:03.498422 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=7bc1e453939f8b48d5e71fdc0917092d57c0ca1bdc4d4f1b64195d1d0408f035 I0428 15:53:03.498557 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=ac089aa60e3607a02b89f57888305fda4ff1e864d97c282ce055548b654176c3 I0428 15:53:03.498570 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 87.251802ms to process 2 records I0428 15:53:03.498663 1 tasks_processing.go:74] worker 12 stopped. I0428 15:53:03.499314 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=c71f30df65c36d95ba04a2625d5203623473d286da26ee864308f2d7e5ad3ee2 I0428 15:53:03.499651 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=652a816e9faa81f1c03b6ee985cab77c2d6536da474e880ca91e24fcfa2ee8d0 I0428 15:53:03.499666 1 gather.go:177] gatherer "clusterconfig" function "crds" took 87.473023ms to process 2 records I0428 15:53:03.502618 1 tasks_processing.go:74] worker 22 stopped. I0428 15:53:03.502651 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0428 15:53:03.502660 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 93.953929ms to process 1 records E0428 15:53:03.505302 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27d38db5bf-db3f-4e06-81b8-af34d6438ff4%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:60016->172.30.0.10:53: read: connection refused I0428 15:53:03.505315 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27d38db5bf-db3f-4e06-81b8-af34d6438ff4%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:60016->172.30.0.10:53: read: connection refused I0428 15:53:03.509969 1 tasks_processing.go:74] worker 18 stopped. E0428 15:53:03.509983 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0428 15:53:03.509989 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0428 15:53:03.509993 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0428 15:53:03.510002 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0428 15:53:03.510030 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0428 15:53:03.510037 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0428 15:53:03.510040 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0428 15:53:03.510044 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0428 15:53:03.510051 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0428 15:53:03.510082 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0428 15:53:03.510089 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0428 15:53:03.510095 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 101.167256ms to process 7 records W0428 15:53:03.513412 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:59228->172.30.0.10:53: read: connection refused I0428 15:53:03.513426 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:59228->172.30.0.10:53: read: connection refused I0428 15:53:03.514122 1 tasks_processing.go:74] worker 5 stopped. E0428 15:53:03.514137 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0428 15:53:03.514144 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pv84h5m32co64u3vo8f52iq6s12dnq0-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pv84h5m32co64u3vo8f52iq6s12dnq0-primary-cert-bundle-secret" not found I0428 15:53:03.514213 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=9c59ebf156526daa3e6d03dc69d4d2b4dc24284621b97a510f956809027e9be7 I0428 15:53:03.514226 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 103.121289ms to process 1 records I0428 15:53:03.571381 1 base_controller.go:82] Caches are synced for LoggingSyncer I0428 15:53:03.571413 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0428 15:53:03.892913 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0428 15:53:03.892929 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0428 15:53:03.893761 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-p6msh pod in namespace openshift-dns (previous: false). I0428 15:53:04.094395 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-p6msh pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-p6msh\" is waiting to start: ContainerCreating" I0428 15:53:04.094413 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-p6msh\" is waiting to start: ContainerCreating" I0428 15:53:04.094421 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-p6msh pod in namespace openshift-dns (previous: false). I0428 15:53:04.273265 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-p6msh pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-p6msh\" is waiting to start: ContainerCreating" I0428 15:53:04.273287 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-p6msh\" is waiting to start: ContainerCreating" I0428 15:53:04.273318 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-vvshq pod in namespace openshift-dns (previous: false). W0428 15:53:04.463868 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0428 15:53:04.483707 1 tasks_processing.go:74] worker 36 stopped. I0428 15:53:04.483724 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 1.074517384s to process 0 records I0428 15:53:04.496420 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-vvshq pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-vvshq\" is waiting to start: ContainerCreating" I0428 15:53:04.496437 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-vvshq\" is waiting to start: ContainerCreating" I0428 15:53:04.496446 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-vvshq pod in namespace openshift-dns (previous: false). I0428 15:53:04.673700 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-vvshq pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-vvshq\" is waiting to start: ContainerCreating" I0428 15:53:04.673717 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-vvshq\" is waiting to start: ContainerCreating" I0428 15:53:04.673745 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-xhvnd pod in namespace openshift-dns (previous: false). I0428 15:53:04.893261 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xhvnd pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-xhvnd\" is waiting to start: ContainerCreating" I0428 15:53:04.893282 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-xhvnd\" is waiting to start: ContainerCreating" I0428 15:53:04.893293 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-xhvnd pod in namespace openshift-dns (previous: false). I0428 15:53:04.904102 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0428 15:53:05.072930 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xhvnd pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-xhvnd\" is waiting to start: ContainerCreating" I0428 15:53:05.072949 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-xhvnd\" is waiting to start: ContainerCreating" I0428 15:53:05.072962 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-68jp4 pod in namespace openshift-dns (previous: false). I0428 15:53:05.274781 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0428 15:53:05.274799 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-7f775 pod in namespace openshift-dns (previous: false). W0428 15:53:05.463256 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0428 15:53:05.472781 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0428 15:53:05.472798 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-9nj8r pod in namespace openshift-dns (previous: false). I0428 15:53:05.507357 1 tasks_processing.go:74] worker 28 stopped. I0428 15:53:05.507432 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=5124c3adc03cbca5374e4102afb026134d81d7fded7315e4e8f5d7e550dd6e17 I0428 15:53:05.507495 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=0aa68589369b53e78385f42a34b617a4bfcc917f6fe417d6d223a0505f422c22 I0428 15:53:05.507544 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0428 15:53:05.507583 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=10d3bcc43dfe4a405328e5712dfe740a794d693eb5e68bf05df5b2fdfcaf807c I0428 15:53:05.507622 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0428 15:53:05.507649 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=556cf2141496c688f092b4ca5a0a97ff6519276c1e9dc2b65f17b574db8daeb8 I0428 15:53:05.507880 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=233dec9d29e301143beb0352c6e712b38e516c5590e41d6bf96ead1bb38a66f9 I0428 15:53:05.507969 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=ae5371ddfd2f5b8ce0322e9ef1afeaef6feb3e80ed5ab9aaed348383be8c1eb8 I0428 15:53:05.508007 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=881ae5a71ed782d5124cb15dd49adf602ee71c4bf7b63fd8c870b8846abcd87a I0428 15:53:05.508120 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=a629804b4c1476d5f2b3832254b7295228d9b35aaebfc0c2d6fb021856ecefae I0428 15:53:05.508298 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0428 15:53:05.508397 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=6c95ceab2d66cd2f3b2e7771f84f90dfb051538f328682b55b4d7254b4b1a4b5 I0428 15:53:05.508424 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0428 15:53:05.508467 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=d491e01d209ee8ba0b52c39c4e6f058fda5ed8bce7e8e423c02d6b27bd9058fd I0428 15:53:05.508528 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0428 15:53:05.508564 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=8caeb6c67b6169849dca736c300c944d1dc14cb1e034f90b00436e49b9008502 I0428 15:53:05.508611 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0428 15:53:05.508643 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=dcf1326393db4b82586233b7e58b61da2630deac123e873a6eae8d81aa78003f I0428 15:53:05.508823 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=8db84dc005b7f7e29fbbcea97f76d6feaa71c355732d8b27974ad66863c55c3b I0428 15:53:05.508838 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0428 15:53:05.508852 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0428 15:53:05.508905 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0428 15:53:05.508945 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=8c3c099cb5995191b990e7c9a7ad5adc2175bcab781fcd0770c74909285ee5a1 I0428 15:53:05.508981 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=3a1dbe8ff595c03d2cbbad1d9d00aad9ff468288caaf637e5387aa789ec6534b I0428 15:53:05.508995 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0428 15:53:05.509023 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=07ee999252c2a3c90c9d6d49cab5a42edefad469062a0f8693b189090dd279cb I0428 15:53:05.509038 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0428 15:53:05.509064 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=95c9654cbc218f720422c1b9fb96772bc7fca96a4bacd39639baa5e0dcd42d34 I0428 15:53:05.509117 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=37133fd65c5adfacb0cc8d0c60080310166dfbf7cfa35bf56b8446aed58f55c3 I0428 15:53:05.509140 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=1783055dc1d791cc72ddab5fcb20dd27e32583e570e10b1af3e84a0af424d3c0 I0428 15:53:05.509161 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=5422b260601ff9666e7d66a59ed31be7b98da8c1832867209f0d33ad3dd89393 I0428 15:53:05.509266 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=6c1b7faee3c573c378c31a0946a42e740a64b4bc07e0f5fc93734d3bcfd952d6 I0428 15:53:05.509327 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0428 15:53:05.509601 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=8b5e3f14df2ceaf19ad691fbe3cca25d70d4d9ade46c4fb9a79b626fd664318d I0428 15:53:05.509619 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0428 15:53:05.509629 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0428 15:53:05.509637 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.098570018s to process 36 records I0428 15:53:05.673654 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0428 15:53:05.673744 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-66754dc85d-g4zq2 pod in namespace openshift-image-registry (previous: false). I0428 15:53:05.873651 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-66754dc85d-g4zq2 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-66754dc85d-g4zq2\" is waiting to start: ContainerCreating" I0428 15:53:05.873670 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-66754dc85d-g4zq2\" is waiting to start: ContainerCreating" I0428 15:53:05.873704 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-79c9f4bf65-4w29d pod in namespace openshift-image-registry (previous: false). I0428 15:53:06.073227 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-79c9f4bf65-4w29d pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-79c9f4bf65-4w29d\" is waiting to start: ContainerCreating" I0428 15:53:06.073245 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-79c9f4bf65-4w29d\" is waiting to start: ContainerCreating" I0428 15:53:06.073294 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-79c9f4bf65-7xbr5 pod in namespace openshift-image-registry (previous: false). I0428 15:53:06.272296 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-79c9f4bf65-7xbr5 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-79c9f4bf65-7xbr5\" is waiting to start: ContainerCreating" I0428 15:53:06.272313 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-79c9f4bf65-7xbr5\" is waiting to start: ContainerCreating" I0428 15:53:06.272324 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-cp2k6 pod in namespace openshift-image-registry (previous: false). W0428 15:53:06.463394 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0428 15:53:06.475060 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0428 15:53:06.475078 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-fq8k5 pod in namespace openshift-image-registry (previous: false). I0428 15:53:06.672871 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0428 15:53:06.672887 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-lx7k8 pod in namespace openshift-image-registry (previous: false). I0428 15:53:06.872883 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0428 15:53:06.872902 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-78c8665cc5-bj6wc pod in namespace openshift-ingress (previous: false). I0428 15:53:07.073509 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-78c8665cc5-bj6wc pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-78c8665cc5-bj6wc\" is waiting to start: ContainerCreating" I0428 15:53:07.073527 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-78c8665cc5-bj6wc\" is waiting to start: ContainerCreating" I0428 15:53:07.073537 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-965cbf5d9-dc4n7 pod in namespace openshift-ingress (previous: false). I0428 15:53:07.272480 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-965cbf5d9-dc4n7 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-965cbf5d9-dc4n7\" is waiting to start: ContainerCreating" I0428 15:53:07.272498 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-965cbf5d9-dc4n7\" is waiting to start: ContainerCreating" I0428 15:53:07.272508 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-965cbf5d9-rwtxh pod in namespace openshift-ingress (previous: false). W0428 15:53:07.464214 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0428 15:53:07.487925 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-965cbf5d9-rwtxh pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-965cbf5d9-rwtxh\" is waiting to start: ContainerCreating" I0428 15:53:07.487942 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-965cbf5d9-rwtxh\" is waiting to start: ContainerCreating" I0428 15:53:07.487976 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-bwlr9 pod in namespace openshift-ingress-canary (previous: false). I0428 15:53:07.675859 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-bwlr9 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-bwlr9\" is waiting to start: ContainerCreating" I0428 15:53:07.675876 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-bwlr9\" is waiting to start: ContainerCreating" I0428 15:53:07.675906 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-kfnl6 pod in namespace openshift-ingress-canary (previous: false). I0428 15:53:07.872318 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-kfnl6 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-kfnl6\" is waiting to start: ContainerCreating" I0428 15:53:07.872337 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-kfnl6\" is waiting to start: ContainerCreating" I0428 15:53:07.872366 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-rplqk pod in namespace openshift-ingress-canary (previous: false). I0428 15:53:08.073365 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-rplqk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rplqk\" is waiting to start: ContainerCreating" I0428 15:53:08.073383 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-rplqk\" is waiting to start: ContainerCreating" I0428 15:53:08.073399 1 tasks_processing.go:74] worker 63 stopped. I0428 15:53:08.073489 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=265819f9af76dd2d0c103b1fa1169480ac8662c54c057c972e881ff196c23d3a I0428 15:53:08.073542 1 recorder.go:75] Recording events/openshift-dns with fingerprint=b338c1d71acd2cbbf87c796e71e6244134da20514a42b90921cc4bb34343b2ad I0428 15:53:08.073631 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=de26ced2910181e7cdcda5ae4d83be3e3bcaa5a5bb0fc2884a19451495b610d3 I0428 15:53:08.073658 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=e4b19878e3a7162840728cee59029f7982f6bcf8e639eefb54d02708fbb40d1b I0428 15:53:08.073705 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=5349e7d4412747cd23cf2468859ad24df515602a39806359bc18a67985d75570 I0428 15:53:08.073722 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=b4f8d45490288fa969ed91efe8419802a59efd94971246c92cabcfb787a8aa54 I0428 15:53:08.073853 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-p6msh with fingerprint=919e9e7962b74e4f7a1490d327fad3fc44f3b5164cff33d664da66f20b228e54 I0428 15:53:08.073939 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-vvshq with fingerprint=2033439c6de52c85fa11900f8a1325f1331583045d86eb0f95890c83f850a66a I0428 15:53:08.074013 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-xhvnd with fingerprint=1dd24d8a9a504bb591f20093ab4dbe8866bbccbcd75e5a01e7cdbb9d5cd29b21 I0428 15:53:08.074123 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-66754dc85d-g4zq2 with fingerprint=3be75e2a663224a6ec824267f564361f8e5e81a70a07a250db88614b8bdec35d I0428 15:53:08.074234 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-79c9f4bf65-4w29d with fingerprint=b23a4129c4f7655a72eaaab5184c78d10faaa3136c6863b72ac5593090dbe913 I0428 15:53:08.074322 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-79c9f4bf65-7xbr5 with fingerprint=fd07d431f763fc440918ef86dca6760055d3ceefe57487ab3fdc8b72fa776618 I0428 15:53:08.074380 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-bwlr9 with fingerprint=ddc31ff28e4a153cb0a2099fceb4103918c2ead2abd1f6ded9698ef7bc3ea2a8 I0428 15:53:08.074433 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-kfnl6 with fingerprint=4a83574ce63c040e68e2fab88a3b598f6f87c6aa60875eb835850b5c9f6cf059 I0428 15:53:08.074486 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-rplqk with fingerprint=24a213439322ade877d2c4248c536b18d27d47cba964cc341c2be2d6c13240db I0428 15:53:08.074495 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.664120229s to process 15 records W0428 15:53:08.460713 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0428 15:53:08.460759 1 tasks_processing.go:74] worker 26 stopped. E0428 15:53:08.460769 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0428 15:53:08.460782 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0428 15:53:08.460797 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0428 15:53:08.460806 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.050944118s to process 1 records I0428 15:53:14.171535 1 configmapobserver.go:84] configmaps "insights-config" not found I0428 15:53:15.867752 1 tasks_processing.go:74] worker 29 stopped. I0428 15:53:15.867795 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0428 15:53:15.867810 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.458912517s to process 1 records I0428 15:53:16.618065 1 tasks_processing.go:74] worker 7 stopped. I0428 15:53:16.618443 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=b3a2c4fa45ed7ae58b9c4093be9a56682fdfea23d7350a26feed0cd7c1d1f420 I0428 15:53:16.618461 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.207976132s to process 1 records E0428 15:53:16.618517 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.21s with: function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0428 15:53:16.619625 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0428 15:53:16.619641 1 periodic.go:209] Running workloads gatherer I0428 15:53:16.619656 1 tasks_processing.go:45] number of workers: 2 I0428 15:53:16.619665 1 tasks_processing.go:69] worker 1 listening for tasks. I0428 15:53:16.619671 1 tasks_processing.go:71] worker 1 working on workload_info task. I0428 15:53:16.619676 1 tasks_processing.go:69] worker 0 listening for tasks. I0428 15:53:16.619703 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0428 15:53:16.643679 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0428 15:53:16.647202 1 tasks_processing.go:74] worker 0 stopped. I0428 15:53:16.647219 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 27.468163ms to process 0 records I0428 15:53:16.651281 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (8ms) I0428 15:53:16.659373 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0428 15:53:16.667101 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0428 15:53:16.674774 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (8ms) I0428 15:53:16.682229 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (7ms) I0428 15:53:16.690397 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (8ms) I0428 15:53:16.699381 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (9ms) I0428 15:53:16.715460 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (16ms) I0428 15:53:16.725406 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (10ms) I0428 15:53:16.736403 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (11ms) I0428 15:53:16.753435 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (17ms) I0428 15:53:16.857796 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (104ms) I0428 15:53:16.951857 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (94ms) I0428 15:53:17.051814 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0428 15:53:17.152351 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (101ms) I0428 15:53:17.252037 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0428 15:53:17.351800 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0428 15:53:17.454927 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (103ms) I0428 15:53:17.551935 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (97ms) I0428 15:53:17.651879 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0428 15:53:17.670685 1 configmapobserver.go:84] configmaps "insights-config" not found I0428 15:53:17.752518 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0428 15:53:17.852721 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0428 15:53:17.871490 1 configmapobserver.go:84] configmaps "insights-config" not found I0428 15:53:17.952257 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0428 15:53:18.054144 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (102ms) I0428 15:53:18.152023 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (98ms) I0428 15:53:18.254740 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (103ms) I0428 15:53:18.358931 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (104ms) I0428 15:53:18.452037 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (93ms) I0428 15:53:18.552362 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0428 15:53:18.652640 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0428 15:53:18.752809 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0428 15:53:18.853426 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0428 15:53:18.853456 1 tasks_processing.go:74] worker 1 stopped. E0428 15:53:18.853467 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0428 15:53:18.853703 1 recorder.go:75] Recording config/workload_info with fingerprint=c0148e2f13a017d70f1b0738aafedbeb31e206128c3011d4a92f7afbc61cef03 I0428 15:53:18.853719 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.233778634s to process 1 records E0428 15:53:18.853743 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.234s with: function \"workload_info\" failed with an error" I0428 15:53:18.854842 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0428 15:53:18.854855 1 periodic.go:209] Running conditional gatherer I0428 15:53:18.861670 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0428 15:53:18.868303 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.10:46853->172.30.0.10:53: read: connection refused E0428 15:53:18.868567 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0428 15:53:18.868625 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0428 15:53:18.874231 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0428 15:53:18.874246 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874250 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874255 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874258 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874261 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874265 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874267 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874270 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0428 15:53:18.874273 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0428 15:53:18.874289 1 tasks_processing.go:45] number of workers: 3 I0428 15:53:18.874300 1 tasks_processing.go:69] worker 2 listening for tasks. I0428 15:53:18.874304 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0428 15:53:18.874311 1 tasks_processing.go:69] worker 0 listening for tasks. I0428 15:53:18.874325 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0428 15:53:18.874324 1 tasks_processing.go:69] worker 1 listening for tasks. I0428 15:53:18.874336 1 tasks_processing.go:74] worker 1 stopped. I0428 15:53:18.874339 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0428 15:53:18.874383 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0428 15:53:18.874396 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.03µs to process 1 records I0428 15:53:18.874427 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0428 15:53:18.874448 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.221µs to process 1 records I0428 15:53:18.874456 1 tasks_processing.go:74] worker 0 stopped. I0428 15:53:18.874595 1 tasks_processing.go:74] worker 2 stopped. I0428 15:53:18.874607 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 247.258µs to process 0 records I0428 15:53:18.874627 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.10:46853->172.30.0.10:53: read: connection refused I0428 15:53:18.874652 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0428 15:53:18.897252 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=ead328cfdb51fe8705cd31236bbc088eff4f0d45077018f30a7cd034abc2e432 I0428 15:53:18.897374 1 diskrecorder.go:70] Writing 111 records to /var/lib/insights-operator/insights-2026-04-28-155318.tar.gz I0428 15:53:18.905740 1 diskrecorder.go:51] Wrote 111 records to disk in 8ms I0428 15:53:18.905776 1 periodic.go:278] Gathering cluster info every 2h0m0s I0428 15:53:18.905799 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0428 15:53:24.695007 1 configmapobserver.go:84] configmaps "insights-config" not found I0428 15:54:22.633496 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="5af560c0ad84ee149a9de4c205178e002ef7b42be803267c298a9e155507892c") W0428 15:54:22.633531 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0428 15:54:22.633574 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="c00be82704fe04e116e328461417709a3d77bc15b5a055fab9335cefc52a47cc") I0428 15:54:22.633590 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0428 15:54:22.633633 1 base_controller.go:181] Shutting down ConfigController ...