W0424 07:14:43.196247 1 cmd.go:257] Using insecure, self-signed certificates I0424 07:14:43.754741 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 07:14:43.755050 1 observer_polling.go:159] Starting file observer I0424 07:14:44.204922 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0424 07:14:44.205109 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0424 07:14:44.205662 1 secure_serving.go:57] Forcing use of http/1.1 only W0424 07:14:44.205683 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0424 07:14:44.205690 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0424 07:14:44.205697 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0424 07:14:44.205702 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0424 07:14:44.205706 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0424 07:14:44.205712 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0424 07:14:44.205694 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0424 07:14:44.209082 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0424 07:14:44.209090 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"eef8bf40-80c0-458f-b4c8-08e09a14099a", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0424 07:14:44.211030 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0424 07:14:44.211050 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0424 07:14:44.211048 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0424 07:14:44.211043 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0424 07:14:44.211071 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0424 07:14:44.211065 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0424 07:14:44.211383 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3536929452/tls.crt::/tmp/serving-cert-3536929452/tls.key" I0424 07:14:44.211635 1 secure_serving.go:213] Serving securely on [::]:8443 I0424 07:14:44.211661 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0424 07:14:44.214033 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0424 07:14:44.214057 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0424 07:14:44.214158 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0424 07:14:44.219654 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 07:14:44.219670 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0424 07:14:44.224092 1 secretconfigobserver.go:119] support secret does not exist I0424 07:14:44.228754 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0424 07:14:44.234063 1 secretconfigobserver.go:119] support secret does not exist I0424 07:14:44.235716 1 recorder.go:161] Pruning old reports every 5h57m25s, max age is 288h0m0s I0424 07:14:44.240656 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0424 07:14:44.240675 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0424 07:14:44.240682 1 insightsreport.go:296] Starting report retriever I0424 07:14:44.240684 1 periodic.go:209] Running clusterconfig gatherer I0424 07:14:44.240686 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0424 07:14:44.240674 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0424 07:14:44.240741 1 tasks_processing.go:45] number of workers: 64 I0424 07:14:44.240792 1 tasks_processing.go:69] worker 3 listening for tasks. I0424 07:14:44.240798 1 tasks_processing.go:69] worker 13 listening for tasks. I0424 07:14:44.240802 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 07:14:44.240807 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 07:14:44.240808 1 tasks_processing.go:69] worker 9 listening for tasks. I0424 07:14:44.240814 1 tasks_processing.go:69] worker 11 listening for tasks. I0424 07:14:44.240819 1 tasks_processing.go:71] worker 11 working on operators task. I0424 07:14:44.240818 1 tasks_processing.go:69] worker 4 listening for tasks. I0424 07:14:44.240820 1 tasks_processing.go:69] worker 14 listening for tasks. I0424 07:14:44.240823 1 tasks_processing.go:69] worker 2 listening for tasks. I0424 07:14:44.240808 1 tasks_processing.go:69] worker 8 listening for tasks. I0424 07:14:44.240813 1 tasks_processing.go:71] worker 1 working on silenced_alerts task. I0424 07:14:44.240837 1 tasks_processing.go:69] worker 42 listening for tasks. I0424 07:14:44.240838 1 tasks_processing.go:69] worker 41 listening for tasks. I0424 07:14:44.240840 1 tasks_processing.go:69] worker 6 listening for tasks. I0424 07:14:44.240843 1 tasks_processing.go:69] worker 43 listening for tasks. I0424 07:14:44.240839 1 tasks_processing.go:69] worker 10 listening for tasks. I0424 07:14:44.240849 1 tasks_processing.go:69] worker 44 listening for tasks. I0424 07:14:44.240850 1 tasks_processing.go:69] worker 17 listening for tasks. I0424 07:14:44.240852 1 tasks_processing.go:71] worker 10 working on feature_gates task. I0424 07:14:44.240854 1 tasks_processing.go:69] worker 45 listening for tasks. I0424 07:14:44.240859 1 tasks_processing.go:69] worker 18 listening for tasks. I0424 07:14:44.240861 1 tasks_processing.go:69] worker 56 listening for tasks. I0424 07:14:44.240859 1 tasks_processing.go:69] worker 57 listening for tasks. I0424 07:14:44.240866 1 tasks_processing.go:69] worker 58 listening for tasks. I0424 07:14:44.240869 1 tasks_processing.go:69] worker 30 listening for tasks. I0424 07:14:44.240866 1 tasks_processing.go:71] worker 4 working on machine_sets task. I0424 07:14:44.240876 1 tasks_processing.go:69] worker 59 listening for tasks. I0424 07:14:44.240876 1 tasks_processing.go:71] worker 8 working on crds task. I0424 07:14:44.240880 1 tasks_processing.go:69] worker 47 listening for tasks. I0424 07:14:44.240876 1 tasks_processing.go:69] worker 29 listening for tasks. I0424 07:14:44.240884 1 tasks_processing.go:71] worker 6 working on overlapping_namespace_uids task. I0424 07:14:44.240890 1 tasks_processing.go:69] worker 48 listening for tasks. I0424 07:14:44.240893 1 tasks_processing.go:69] worker 19 listening for tasks. I0424 07:14:44.240829 1 tasks_processing.go:69] worker 12 listening for tasks. I0424 07:14:44.240901 1 tasks_processing.go:69] worker 33 listening for tasks. I0424 07:14:44.240892 1 tasks_processing.go:69] worker 32 listening for tasks. I0424 07:14:44.240909 1 tasks_processing.go:69] worker 21 listening for tasks. I0424 07:14:44.240911 1 tasks_processing.go:69] worker 34 listening for tasks. I0424 07:14:44.240826 1 tasks_processing.go:69] worker 15 listening for tasks. I0424 07:14:44.240914 1 tasks_processing.go:69] worker 37 listening for tasks. I0424 07:14:44.240919 1 tasks_processing.go:69] worker 52 listening for tasks. I0424 07:14:44.240920 1 tasks_processing.go:69] worker 63 listening for tasks. I0424 07:14:44.240873 1 tasks_processing.go:71] worker 2 working on qemu_kubevirt_launcher_logs task. I0424 07:14:44.240927 1 tasks_processing.go:69] worker 60 listening for tasks. I0424 07:14:44.240931 1 tasks_processing.go:71] worker 63 working on openstack_version task. I0424 07:14:44.240940 1 tasks_processing.go:69] worker 62 listening for tasks. I0424 07:14:44.240936 1 tasks_processing.go:71] worker 34 working on networks task. I0424 07:14:44.240941 1 tasks_processing.go:69] worker 25 listening for tasks. I0424 07:14:44.240949 1 tasks_processing.go:69] worker 26 listening for tasks. I0424 07:14:44.240947 1 tasks_processing.go:69] worker 61 listening for tasks. I0424 07:14:44.240958 1 tasks_processing.go:71] worker 26 working on config_maps task. I0424 07:14:44.240956 1 tasks_processing.go:69] worker 54 listening for tasks. I0424 07:14:44.240960 1 tasks_processing.go:69] worker 28 listening for tasks. I0424 07:14:44.240882 1 tasks_processing.go:71] worker 41 working on cost_management_metrics_configs task. I0424 07:14:44.240977 1 tasks_processing.go:69] worker 53 listening for tasks. I0424 07:14:44.240979 1 tasks_processing.go:71] worker 28 working on openstack_controlplanes task. I0424 07:14:44.240984 1 tasks_processing.go:71] worker 53 working on image task. I0424 07:14:44.240981 1 tasks_processing.go:69] worker 27 listening for tasks. I0424 07:14:44.240933 1 tasks_processing.go:71] worker 60 working on active_alerts task. I0424 07:14:44.241001 1 tasks_processing.go:71] worker 13 working on dvo_metrics task. I0424 07:14:44.240875 1 tasks_processing.go:69] worker 35 listening for tasks. I0424 07:14:44.240884 1 tasks_processing.go:69] worker 31 listening for tasks. I0424 07:14:44.240891 1 tasks_processing.go:69] worker 38 listening for tasks. I0424 07:14:44.240898 1 tasks_processing.go:69] worker 49 listening for tasks. I0424 07:14:44.240902 1 tasks_processing.go:69] worker 20 listening for tasks. I0424 07:14:44.240905 1 tasks_processing.go:69] worker 50 listening for tasks. I0424 07:14:44.240905 1 tasks_processing.go:69] worker 36 listening for tasks. I0424 07:14:44.240912 1 tasks_processing.go:69] worker 51 listening for tasks. I0424 07:14:44.240918 1 tasks_processing.go:69] worker 22 listening for tasks. I0424 07:14:44.240849 1 tasks_processing.go:69] worker 7 listening for tasks. I0424 07:14:44.240925 1 tasks_processing.go:69] worker 23 listening for tasks. I0424 07:14:44.241028 1 tasks_processing.go:71] worker 52 working on ingress task. I0424 07:14:44.241034 1 tasks_processing.go:71] worker 54 working on openstack_dataplanedeployments task. I0424 07:14:44.241043 1 tasks_processing.go:71] worker 0 working on image_pruners task. I0424 07:14:44.240926 1 tasks_processing.go:71] worker 37 working on operators_pods_and_events task. I0424 07:14:44.241073 1 tasks_processing.go:71] worker 59 working on storage_classes task. I0424 07:14:44.241124 1 tasks_processing.go:71] worker 12 working on authentication task. I0424 07:14:44.240858 1 tasks_processing.go:71] worker 43 working on mutating_webhook_configurations task. I0424 07:14:44.241141 1 tasks_processing.go:71] worker 27 working on metrics task. I0424 07:14:44.240946 1 tasks_processing.go:71] worker 62 working on certificate_signing_requests task. I0424 07:14:44.241914 1 tasks_processing.go:71] worker 35 working on sap_datahubs task. I0424 07:14:44.241967 1 tasks_processing.go:71] worker 15 working on container_runtime_configs task. I0424 07:14:44.242172 1 tasks_processing.go:69] worker 39 listening for tasks. I0424 07:14:44.242190 1 tasks_processing.go:71] worker 39 working on image_registries task. I0424 07:14:44.242272 1 tasks_processing.go:69] worker 40 listening for tasks. I0424 07:14:44.242276 1 tasks_processing.go:71] worker 56 working on nodes task. I0424 07:14:44.242377 1 tasks_processing.go:71] worker 44 working on version task. I0424 07:14:44.242431 1 tasks_processing.go:71] worker 40 working on storage_cluster task. I0424 07:14:44.242467 1 tasks_processing.go:71] worker 31 working on nodenetworkstates task. I0424 07:14:44.242541 1 tasks_processing.go:69] worker 55 listening for tasks. I0424 07:14:44.242559 1 tasks_processing.go:71] worker 55 working on machine_autoscalers task. I0424 07:14:44.240874 1 tasks_processing.go:69] worker 46 listening for tasks. I0424 07:14:44.240879 1 tasks_processing.go:71] worker 42 working on container_images task. I0424 07:14:44.242664 1 tasks_processing.go:71] worker 46 working on jaegers task. I0424 07:14:44.242743 1 tasks_processing.go:71] worker 29 working on install_plans task. I0424 07:14:44.242923 1 tasks_processing.go:71] worker 48 working on monitoring_persistent_volumes task. I0424 07:14:44.242936 1 tasks_processing.go:71] worker 38 working on lokistack task. I0424 07:14:44.243089 1 tasks_processing.go:71] worker 32 working on schedulers task. I0424 07:14:44.240870 1 tasks_processing.go:71] worker 14 working on aggregated_monitoring_cr_names task. I0424 07:14:44.241143 1 tasks_processing.go:71] worker 21 working on validating_webhook_configurations task. I0424 07:14:44.243147 1 tasks_processing.go:71] worker 33 working on openshift_logging task. I0424 07:14:44.243176 1 tasks_processing.go:71] worker 20 working on pdbs task. I0424 07:14:44.240966 1 tasks_processing.go:71] worker 61 working on infrastructures task. I0424 07:14:44.241066 1 tasks_processing.go:71] worker 3 working on clusterroles task. I0424 07:14:44.240932 1 tasks_processing.go:69] worker 24 listening for tasks. I0424 07:14:44.240814 1 tasks_processing.go:71] worker 9 working on sap_pods task. W0424 07:14:44.240865 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 07:14:44.240832 1 tasks_processing.go:69] worker 16 listening for tasks. I0424 07:14:44.240954 1 tasks_processing.go:71] worker 25 working on support_secret task. I0424 07:14:44.241819 1 tasks_processing.go:71] worker 23 working on proxies task. W0424 07:14:44.241820 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory W0424 07:14:44.241019 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 07:14:44.243220 1 tasks_processing.go:71] worker 27 working on node_logs task. I0424 07:14:44.240834 1 tasks_processing.go:69] worker 5 listening for tasks. I0424 07:14:44.243366 1 tasks_processing.go:71] worker 18 working on machines task. I0424 07:14:44.243369 1 tasks_processing.go:71] worker 22 working on olm_operators task. I0424 07:14:44.243401 1 tasks_processing.go:71] worker 30 working on service_accounts task. I0424 07:14:44.243413 1 tasks_processing.go:71] worker 19 working on sap_config task. I0424 07:14:44.243364 1 tasks_processing.go:71] worker 49 working on openstack_dataplanenodesets task. I0424 07:14:44.243553 1 tasks_processing.go:71] worker 16 working on tsdb_status task. I0424 07:14:44.243565 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 1.48225ms to process 0 records I0424 07:14:44.243580 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 2.216828ms to process 0 records W0424 07:14:44.243585 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 07:14:44.243590 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 2.38021ms to process 0 records I0424 07:14:44.243598 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 27.697µs to process 0 records I0424 07:14:44.243603 1 tasks_processing.go:71] worker 51 working on oauths task. I0424 07:14:44.243604 1 tasks_processing.go:71] worker 50 working on machine_configs task. I0424 07:14:44.243687 1 tasks_processing.go:74] worker 16 stopped. I0424 07:14:44.243721 1 tasks_processing.go:71] worker 57 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0424 07:14:44.243773 1 tasks_processing.go:71] worker 47 working on machine_config_pools task. I0424 07:14:44.244212 1 tasks_processing.go:71] worker 36 working on ceph_cluster task. I0424 07:14:44.244221 1 tasks_processing.go:71] worker 45 working on pod_network_connectivity_checks task. I0424 07:14:44.244250 1 tasks_processing.go:74] worker 60 stopped. I0424 07:14:44.244518 1 tasks_processing.go:74] worker 1 stopped. I0424 07:14:44.244700 1 tasks_processing.go:74] worker 41 stopped. I0424 07:14:44.244717 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 3.717537ms to process 0 records I0424 07:14:44.244915 1 tasks_processing.go:74] worker 4 stopped. I0424 07:14:44.244931 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 4.031985ms to process 0 records I0424 07:14:44.245103 1 tasks_processing.go:71] worker 17 working on machine_healthchecks task. I0424 07:14:44.245105 1 tasks_processing.go:71] worker 58 working on cluster_apiserver task. I0424 07:14:44.245240 1 tasks_processing.go:71] worker 5 working on ingress_certificates task. I0424 07:14:44.245277 1 tasks_processing.go:71] worker 24 working on openshift_machine_api_events task. I0424 07:14:44.245311 1 tasks_processing.go:71] worker 7 working on nodenetworkconfigurationpolicies task. I0424 07:14:44.245333 1 tasks_processing.go:74] worker 63 stopped. I0424 07:14:44.245350 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 4.390292ms to process 0 records I0424 07:14:44.246203 1 tasks_processing.go:74] worker 54 stopped. I0424 07:14:44.246222 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 5.161181ms to process 0 records I0424 07:14:44.246431 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0424 07:14:44.246451 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0424 07:14:44.246457 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0424 07:14:44.246462 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0424 07:14:44.246480 1 controller.go:489] The operator is still being initialized I0424 07:14:44.246491 1 controller.go:512] The operator is healthy I0424 07:14:44.247119 1 tasks_processing.go:74] worker 40 stopped. I0424 07:14:44.247132 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 4.674975ms to process 0 records I0424 07:14:44.247356 1 tasks_processing.go:74] worker 34 stopped. I0424 07:14:44.247533 1 recorder.go:75] Recording config/network with fingerprint=c3932c72aacf33a3798c9b05959062400ea1256cd7be9b63285077b5f354d924 I0424 07:14:44.247546 1 gather.go:177] gatherer "clusterconfig" function "networks" took 6.397501ms to process 1 records I0424 07:14:44.247730 1 tasks_processing.go:74] worker 10 stopped. I0424 07:14:44.247877 1 recorder.go:75] Recording config/featuregate with fingerprint=3be865d82c436b376cb74610911fc95d9f19ef790e12080b5a8f3fddde411e06 I0424 07:14:44.247889 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 6.866595ms to process 1 records I0424 07:14:44.248166 1 tasks_processing.go:74] worker 31 stopped. I0424 07:14:44.248182 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 5.682267ms to process 0 records I0424 07:14:44.249845 1 tasks_processing.go:74] worker 38 stopped. I0424 07:14:44.249859 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 6.895737ms to process 0 records I0424 07:14:44.254292 1 tasks_processing.go:74] worker 28 stopped. I0424 07:14:44.254307 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 13.303061ms to process 0 records I0424 07:14:44.263411 1 tasks_processing.go:74] worker 45 stopped. E0424 07:14:44.263422 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0424 07:14:44.263429 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 19.17947ms to process 0 records I0424 07:14:44.263486 1 tasks_processing.go:74] worker 18 stopped. E0424 07:14:44.263498 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0424 07:14:44.263505 1 gather.go:177] gatherer "clusterconfig" function "machines" took 20.077265ms to process 0 records I0424 07:14:44.263695 1 tasks_processing.go:74] worker 33 stopped. I0424 07:14:44.263704 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 20.531166ms to process 0 records I0424 07:14:44.263923 1 tasks_processing.go:74] worker 53 stopped. I0424 07:14:44.263996 1 recorder.go:75] Recording config/image with fingerprint=3517e0c8d466f89dbf521545cefc6ae6e9d5a0e84746bce1ec2bba14b253e82e I0424 07:14:44.264006 1 gather.go:177] gatherer "clusterconfig" function "image" took 22.931392ms to process 1 records I0424 07:14:44.265367 1 tasks_processing.go:74] worker 19 stopped. I0424 07:14:44.265375 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 21.945745ms to process 0 records I0424 07:14:44.265457 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=8f3d61dd9a9622eac5637a0cd578cc6eeda4b823fd20065d32474799616d80f7 I0424 07:14:44.265464 1 tasks_processing.go:74] worker 59 stopped. I0424 07:14:44.265476 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=ab20e93dcf44a1bd87497346d6c59b8642f33fe3077dcca8521a520708c2f67d I0424 07:14:44.265482 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 24.288178ms to process 2 records I0424 07:14:44.265580 1 tasks_processing.go:74] worker 52 stopped. I0424 07:14:44.266132 1 recorder.go:75] Recording config/ingress with fingerprint=b0d92bb8eab1876f97bf62c849856ca9d8bda815a762c55290a7fba30db0a7fe I0424 07:14:44.266152 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 24.543017ms to process 1 records I0424 07:14:44.266331 1 tasks_processing.go:74] worker 32 stopped. I0424 07:14:44.266343 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=84c2ff199df0cadbb6a581943859ef17af03a57b3f553de7d4c8ae130ecc2450 I0424 07:14:44.266413 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 22.792212ms to process 1 records I0424 07:14:44.266445 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 21.754223ms to process 0 records I0424 07:14:44.266472 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 23.606691ms to process 0 records I0424 07:14:44.266499 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 23.658898ms to process 0 records I0424 07:14:44.266633 1 recorder.go:75] Recording config/proxy with fingerprint=6400d390b42c19a24743a4d003578d83d5090038e6d9a7d4385a030f31f3e023 I0424 07:14:44.266650 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 23.25309ms to process 1 records I0424 07:14:44.266482 1 tasks_processing.go:74] worker 36 stopped. I0424 07:14:44.266772 1 tasks_processing.go:74] worker 55 stopped. I0424 07:14:44.266814 1 tasks_processing.go:74] worker 46 stopped. I0424 07:14:44.266840 1 tasks_processing.go:74] worker 23 stopped. I0424 07:14:44.267074 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=30ad01d617a8229c45797cb72cb77172f2a15af8cf39ee4322b69960ed02f5aa I0424 07:14:44.267097 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 25.545172ms to process 1 records I0424 07:14:44.267108 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 24.5923ms to process 0 records I0424 07:14:44.267275 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=ecb1be1867b4438354a4cf1a5c6c3edd82c74c780c0642d63e7e6f8d479769f7 I0424 07:14:44.267353 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=9750d0f6953b9c555bc98272d07a04f03da2a9602f75d0941a6e7c397100debb I0424 07:14:44.267406 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=fb3776c3b8fcc5d328fa2f2fbacb8d03998679dd19232d4342f8bccd27729c6e I0424 07:14:44.267436 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 23.473895ms to process 3 records I0424 07:14:44.267468 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 23.806191ms to process 0 records E0424 07:14:44.267496 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0424 07:14:44.267524 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 21.971028ms to process 0 records I0424 07:14:44.267539 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 25.198421ms to process 0 records I0424 07:14:44.267284 1 tasks_processing.go:74] worker 0 stopped. I0424 07:14:44.267354 1 tasks_processing.go:74] worker 20 stopped. I0424 07:14:44.267292 1 tasks_processing.go:74] worker 15 stopped. I0424 07:14:44.267705 1 tasks_processing.go:74] worker 27 stopped. I0424 07:14:44.267794 1 tasks_processing.go:74] worker 17 stopped. I0424 07:14:44.267908 1 tasks_processing.go:74] worker 35 stopped. I0424 07:14:44.267920 1 tasks_processing.go:74] worker 51 stopped. I0424 07:14:44.268052 1 recorder.go:75] Recording config/oauth with fingerprint=96f90694372499ea730ee2d0fcfec4feb29e3ffd2cfa245af566b7cb7a5e9298 I0424 07:14:44.268072 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 23.564108ms to process 1 records I0424 07:14:44.268083 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 23.983719ms to process 0 records I0424 07:14:44.268099 1 tasks_processing.go:74] worker 9 stopped. I0424 07:14:44.268222 1 tasks_processing.go:74] worker 43 stopped. I0424 07:14:44.269401 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=e7d3367b355feb11adc4691f3dee220df8d41210be8974560624668057de1f9f I0424 07:14:44.269471 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=767ff428d0d8af0fb78789dba89da31be170333f6bbcb60c83b89025916c450c I0424 07:14:44.269531 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=5eff8cdf49aa94c955a18a4426337d973ea7c2176177c100e232cc26a42ebc08 I0424 07:14:44.269544 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 26.139947ms to process 3 records E0424 07:14:44.269555 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0424 07:14:44.269564 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 24.154118ms to process 0 records I0424 07:14:44.269572 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 23.629586ms to process 0 records I0424 07:14:44.269578 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 23.972657ms to process 0 records I0424 07:14:44.269586 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 22.341449ms to process 0 records I0424 07:14:44.269609 1 tasks_processing.go:74] worker 47 stopped. I0424 07:14:44.269610 1 tasks_processing.go:74] worker 49 stopped. I0424 07:14:44.269619 1 tasks_processing.go:74] worker 7 stopped. I0424 07:14:44.269624 1 tasks_processing.go:74] worker 25 stopped. I0424 07:14:44.269698 1 tasks_processing.go:74] worker 61 stopped. I0424 07:14:44.270370 1 recorder.go:75] Recording config/infrastructure with fingerprint=8f81d636c729f08f83982356557c2abefed319156aac20ce30b77e32d07c88fa I0424 07:14:44.270388 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 25.0665ms to process 1 records I0424 07:14:44.270396 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 25.414098ms to process 0 records I0424 07:14:44.270454 1 tasks_processing.go:74] worker 48 stopped. I0424 07:14:44.270612 1 tasks_processing.go:74] worker 56 stopped. I0424 07:14:44.270832 1 recorder.go:75] Recording config/node/ip-10-0-0-90.ec2.internal with fingerprint=62b6b2c122f5414f22386b4bfb0242c35462c457cfa997ccf5c346b298869604 I0424 07:14:44.270955 1 recorder.go:75] Recording config/node/ip-10-0-1-236.ec2.internal with fingerprint=77642edceae95d6ad70b334d6431830fe759e35daf1dbd8ca051c95ecd010af2 I0424 07:14:44.271047 1 recorder.go:75] Recording config/node/ip-10-0-2-178.ec2.internal with fingerprint=c62777865c9f6176ac3cb431198142fcbf258527a3a9778fe7b681262a4f9a8d I0424 07:14:44.271061 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 26.926608ms to process 3 records I0424 07:14:44.271078 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0424 07:14:44.271085 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 28.711248ms to process 1 records I0424 07:14:44.271112 1 tasks_processing.go:74] worker 6 stopped. I0424 07:14:44.271189 1 tasks_processing.go:74] worker 12 stopped. I0424 07:14:44.271305 1 recorder.go:75] Recording config/authentication with fingerprint=3fa7229657d637797fe2d5f7a45e0664ec9ea142ec62ba6276d5e1ea9bae6b1f I0424 07:14:44.271338 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 28.589799ms to process 1 records I0424 07:14:44.271416 1 tasks_processing.go:74] worker 58 stopped. I0424 07:14:44.271460 1 recorder.go:75] Recording config/apiserver with fingerprint=b7e39f296b89a4c5269e92534891579f2d66d8622cf6f03890002bab782a7e08 I0424 07:14:44.271470 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 25.294913ms to process 1 records I0424 07:14:44.271475 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 28.793991ms to process 0 records I0424 07:14:44.271556 1 tasks_processing.go:74] worker 62 stopped. I0424 07:14:44.271623 1 tasks_processing.go:74] worker 8 stopped. I0424 07:14:44.272086 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=6fc371cc627d1205d95738051a1e907e7b57e2218d383f4d7113d70dabb2eadb I0424 07:14:44.272425 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=c37b7fbecdea316b430dced5cc4485c63fd3a25e86452c3358edaed40b2c83da I0424 07:14:44.272441 1 gather.go:177] gatherer "clusterconfig" function "crds" took 30.330755ms to process 2 records I0424 07:14:44.272453 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 26.510622ms to process 0 records I0424 07:14:44.272464 1 tasks_processing.go:74] worker 24 stopped. W0424 07:14:44.272629 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 07:14:44.275120 1 gather_logs.go:145] no pods in namespace were found I0424 07:14:44.275138 1 tasks_processing.go:74] worker 2 stopped. I0424 07:14:44.275148 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 34.204966ms to process 0 records I0424 07:14:44.275243 1 tasks_processing.go:74] worker 21 stopped. I0424 07:14:44.275368 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=e2d5f8c2a725715998f3fb1c170a0be221bc74de8055ea3a05174a9876cdc89a I0424 07:14:44.275483 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=61fe35e78a8768e7bd6d0856833464cdda5827ca84f5f1d4597379e8a9b1fe50 I0424 07:14:44.275539 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=70e6471558665113267ddfd7387fda7356b8ac2d8f570f26d87f39d08875a37a I0424 07:14:44.275588 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=3624a330c429357d263bd457b5662dd6659c82bc79cccf56df06c2a17427f23b I0424 07:14:44.275637 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=97c4d58429c1cb72f87e95b96cb1938f8c35216c3d5d0bce4339ec308857cf3d I0424 07:14:44.275670 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=bd6b177400b3d2609668a11dc8a3b19254f4c0bf9d5b60d1a21367bbbc1bd1e6 I0424 07:14:44.275703 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=def0f314ef92b3b8649eedf0c04009ab6f379f79117711673a0e2bce3bcfe26e I0424 07:14:44.275745 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=edde90252011ce924b4f2d3ecea977f1478815bfb95899213418963726204a00 I0424 07:14:44.275797 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=5f62adb8444e34a2e60798d97ce90842d7e2abec52b4251b118f6cfd6e2b5acf I0424 07:14:44.275834 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=6e5d86f1b750dc645a382a4b23a55171ed2547f859caa75b4552bb276ca9f861 I0424 07:14:44.275867 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=6456f0c70ccd978d43cc332b4aa10fed0ffa95cf291b96a3d8b3c9f3a446dbf4 I0424 07:14:44.275879 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 32.086947ms to process 11 records I0424 07:14:44.275898 1 tasks_processing.go:74] worker 39 stopped. I0424 07:14:44.276305 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=dc498441d94332175e43ca9ff69c47a59e16867d92314230749f85dc07267272 I0424 07:14:44.276325 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 33.694946ms to process 1 records I0424 07:14:44.276995 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0424 07:14:44.277048 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0424 07:14:44.277137 1 operator.go:288] started I0424 07:14:44.277159 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0424 07:14:44.299039 1 tasks_processing.go:74] worker 22 stopped. I0424 07:14:44.299108 1 recorder.go:75] Recording config/olm_operators with fingerprint=c64155933a031273294edb659450d787181b22d91ee51c187bcb73d8eb7949a8 I0424 07:14:44.299128 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 55.637893ms to process 1 records I0424 07:14:44.309427 1 tasks_processing.go:74] worker 42 stopped. I0424 07:14:44.310213 1 prometheus_rules.go:88] Prometheus rules successfully created I0424 07:14:44.310926 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-cf9r8 with fingerprint=908648d60a21ce71a16080c4d1b76125a71f8ec1b76d2f9851e2c25dd855b808 I0424 07:14:44.310981 1 recorder.go:75] Recording config/running_containers with fingerprint=d42ae21b355c1325af01ba6ef077920913244dfdccb058001cf8a5b64a94e544 I0424 07:14:44.310991 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 66.808161ms to process 2 records I0424 07:14:44.311075 1 tasks_processing.go:74] worker 3 stopped. I0424 07:14:44.311095 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0424 07:14:44.311096 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0424 07:14:44.311111 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0424 07:14:44.311141 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=f7d6643509097683afea4c5cbb4b4e3b6f8d088590dde1f5dcb1dd1674e9ae87 I0424 07:14:44.311281 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=ef42a9c349eeef65bdb40afe97cfd2ea5a530d64d8471b485325002fac9b9548 I0424 07:14:44.311292 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 67.057714ms to process 2 records I0424 07:14:44.311409 1 tasks_processing.go:74] worker 57 stopped. I0424 07:14:44.311421 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 67.673187ms to process 0 records I0424 07:14:44.314437 1 base_controller.go:82] Caches are synced for ConfigController I0424 07:14:44.314447 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0424 07:14:44.319498 1 tasks_processing.go:74] worker 14 stopped. I0424 07:14:44.319517 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 76.341875ms to process 0 records I0424 07:14:44.336562 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0424 07:14:44.336573 1 controller.go:212] Source scaController *sca.Controller is not ready I0424 07:14:44.336577 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0424 07:14:44.336580 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0424 07:14:44.336583 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0424 07:14:44.336602 1 controller.go:489] The operator is still being initialized I0424 07:14:44.336610 1 controller.go:512] The operator is healthy I0424 07:14:44.343198 1 configmapobserver.go:84] configmaps "insights-config" not found E0424 07:14:44.343945 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27eb7459e0-c436-40e5-9e93-1ddcea221a7d%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:51631->172.30.0.10:53: read: connection refused I0424 07:14:44.343960 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27eb7459e0-c436-40e5-9e93-1ddcea221a7d%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:51631->172.30.0.10:53: read: connection refused I0424 07:14:44.347839 1 tasks_processing.go:74] worker 26 stopped. E0424 07:14:44.347855 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0424 07:14:44.347861 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0424 07:14:44.347864 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0424 07:14:44.347874 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0424 07:14:44.347901 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0424 07:14:44.347917 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0424 07:14:44.347921 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0424 07:14:44.347925 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0424 07:14:44.347963 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0424 07:14:44.347971 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0424 07:14:44.347976 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 106.86514ms to process 7 records I0424 07:14:44.353326 1 tasks_processing.go:74] worker 5 stopped. E0424 07:14:44.353342 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0424 07:14:44.353350 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pscaa57d3d5pmhcunl37lfoah11uk1u-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pscaa57d3d5pmhcunl37lfoah11uk1u-primary-cert-bundle-secret" not found I0424 07:14:44.353402 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=aaa4c4a1163d9db9d16427dee06d8e4c7746f63c06a83bf5fb856a2067d3170e I0424 07:14:44.353414 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 108.071794ms to process 1 records I0424 07:14:44.355243 1 tasks_processing.go:74] worker 44 stopped. I0424 07:14:44.355538 1 recorder.go:75] Recording config/version with fingerprint=49b1c11aa65e5ecf33ebb34da97d9c6c017d781a50d27ad3ebb9fac8c7f9ac08 I0424 07:14:44.355555 1 recorder.go:75] Recording config/id with fingerprint=ac69ab7f2bd7cb5bbc1dcd51cebd05c6e3a74d58a5f0eaffababf7f740612978 I0424 07:14:44.355564 1 gather.go:177] gatherer "clusterconfig" function "version" took 112.849556ms to process 2 records I0424 07:14:44.358866 1 tasks_processing.go:74] worker 50 stopped. I0424 07:14:44.358894 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0424 07:14:44.358906 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 115.159749ms to process 1 records I0424 07:14:44.368124 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0424 07:14:44.371517 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:35629->172.30.0.10:53: read: connection refused I0424 07:14:44.371529 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:35629->172.30.0.10:53: read: connection refused I0424 07:14:44.377244 1 base_controller.go:82] Caches are synced for LoggingSyncer I0424 07:14:44.377255 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0424 07:14:44.377288 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0424 07:14:44.690262 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0424 07:14:44.690278 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0424 07:14:44.691071 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-5dcrq pod in namespace openshift-dns (previous: false). I0424 07:14:44.920961 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-5dcrq pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-5dcrq\" is waiting to start: ContainerCreating" I0424 07:14:44.920977 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-5dcrq\" is waiting to start: ContainerCreating" I0424 07:14:44.920985 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-5dcrq pod in namespace openshift-dns (previous: false). I0424 07:14:45.087941 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-5dcrq pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-5dcrq\" is waiting to start: ContainerCreating" I0424 07:14:45.087961 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-5dcrq\" is waiting to start: ContainerCreating" I0424 07:14:45.087991 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-7dk97 pod in namespace openshift-dns (previous: false). W0424 07:14:45.270204 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 07:14:45.307500 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-7dk97 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-7dk97\" is waiting to start: ContainerCreating" I0424 07:14:45.307516 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-7dk97\" is waiting to start: ContainerCreating" I0424 07:14:45.307524 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-7dk97 pod in namespace openshift-dns (previous: false). I0424 07:14:45.489858 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-7dk97 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-7dk97\" is waiting to start: ContainerCreating" I0424 07:14:45.489873 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-7dk97\" is waiting to start: ContainerCreating" I0424 07:14:45.489902 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-965c7 pod in namespace openshift-dns (previous: false). I0424 07:14:45.711140 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0424 07:14:45.712517 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-965c7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-965c7\" is waiting to start: ContainerCreating" I0424 07:14:45.712528 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-965c7\" is waiting to start: ContainerCreating" I0424 07:14:45.712535 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-965c7 pod in namespace openshift-dns (previous: false). I0424 07:14:45.888245 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-965c7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-965c7\" is waiting to start: ContainerCreating" I0424 07:14:45.888260 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-965c7\" is waiting to start: ContainerCreating" I0424 07:14:45.888270 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-7kgf7 pod in namespace openshift-dns (previous: false). I0424 07:14:46.113925 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 07:14:46.113942 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-ggcfm pod in namespace openshift-dns (previous: false). I0424 07:14:46.114522 1 tasks_processing.go:74] worker 11 stopped. I0424 07:14:46.114578 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=7a59095278a61261b33b50effd7de6fed230dff2fbc258ccf9e4c06ac4301022 I0424 07:14:46.114618 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=9842565f4fdadbdaa39a5fb129a0d41e7935191777d084021ef3f2e650155bd8 I0424 07:14:46.114651 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0424 07:14:46.114678 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=935d456ac165b510aabdf4c1af7a938b150b918162d96c7249e16e199d4d62e4 I0424 07:14:46.114716 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0424 07:14:46.114739 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=a1abeaeae0936d7e5a0b234e084feb11b9aaab93ea5b063e1a51c2071f7e3a90 I0424 07:14:46.114785 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=710910396815c3072cd0a2066f7aacfd1956337643190cc93ca3ebfb0c7f74ee I0424 07:14:46.114811 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=e13d88b1a36205b546517cd93a5bb3f33b1bb804767f8012944801be331ff214 I0424 07:14:46.114825 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=5bdfbedbec50711806198d4b6f5fb1e2962bbc8fdeeb4cc14a3d4b5e84ea799f I0424 07:14:46.114844 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=acc78776358c8a25eaa7f1a1870fc6bb24996052d50e87d7656c8f4239aba154 I0424 07:14:46.114866 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0424 07:14:46.114886 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=abc29888f0b428131716f51acf04e164e113bb5d1a485c3c9658e1324b50b00e I0424 07:14:46.114897 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0424 07:14:46.114914 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=4d9bc5fc8d6674c69f2750d66a6c881f576832ca77f5ef9cdeef7c591a318a24 I0424 07:14:46.114923 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0424 07:14:46.114935 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=580b57217ef23b09d27dadbc7095b656dcfde0f8d75b0cbad1d43ff60d4df91a I0424 07:14:46.114948 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0424 07:14:46.114967 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=1c866b1a2b9743734815c260dbc2a53781ea1b6b2a6f4e21441779e172b028df I0424 07:14:46.115086 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=460fa1c8d7376996fda1b7994cfaf29fa53258ec846b9b5271f53d1d00383fb4 I0424 07:14:46.115099 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0424 07:14:46.115106 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0424 07:14:46.115127 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0424 07:14:46.115148 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=09acb33133176f42f4c61ec020cd7d67113a09f3b800e9685f95b7e46d4db499 I0424 07:14:46.115171 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=dc8fbde2323e4eae621e29bd11c504c33e84b32404ffbd0ca91c9c00130d3e15 I0424 07:14:46.115180 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0424 07:14:46.115196 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=cb3da76c47a49259669f3593025644f8140fe6d97f89fedb351f1bbd99c109d2 I0424 07:14:46.115205 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0424 07:14:46.115218 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=522d45fd347e930f725216d3a14d7cd8d42ba44a564c4c81ef86d33992e75d67 I0424 07:14:46.115232 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=4e8327e444aefa0ccb1336896b2949b6e760fe8cf6c321a6e2e13f4e09e29f6d I0424 07:14:46.115249 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=d66f45e61aeb367690a4ba196a3050ed73a07ef0b1eee54b7426eb73bbefa5be I0424 07:14:46.115264 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=f6ec92b96237bf633bd20293975d192c461cd3f15baa89c9015345054be95fa0 I0424 07:14:46.115277 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=f733e0a324d88307dddfc41d80e58a4e18ae55734f7633ad5555561ac7a3ee85 I0424 07:14:46.115304 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=f90bb89469c6eeb91b47b59cd2e9882b979a22eb2ce4de4322f3b3ed318f8c45 I0424 07:14:46.115320 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0424 07:14:46.115329 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0424 07:14:46.115335 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.873690437s to process 35 records W0424 07:14:46.270085 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 07:14:46.305400 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 07:14:46.305421 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-wv9qg pod in namespace openshift-dns (previous: false). I0424 07:14:46.506098 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 07:14:46.506149 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-65d646d9c9-bgt6w pod in namespace openshift-image-registry (previous: false). I0424 07:14:46.686203 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-65d646d9c9-bgt6w pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-65d646d9c9-bgt6w\" is waiting to start: ContainerCreating" I0424 07:14:46.686222 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-65d646d9c9-bgt6w\" is waiting to start: ContainerCreating" I0424 07:14:46.686268 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-65d646d9c9-dzxmk pod in namespace openshift-image-registry (previous: false). I0424 07:14:46.887857 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-65d646d9c9-dzxmk pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-65d646d9c9-dzxmk\" is waiting to start: ContainerCreating" I0424 07:14:46.887873 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-65d646d9c9-dzxmk\" is waiting to start: ContainerCreating" I0424 07:14:46.887905 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-65fb668976-pxf98 pod in namespace openshift-image-registry (previous: false). I0424 07:14:47.088428 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-65fb668976-pxf98 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-65fb668976-pxf98\" is waiting to start: ContainerCreating" I0424 07:14:47.088444 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-65fb668976-pxf98\" is waiting to start: ContainerCreating" I0424 07:14:47.088455 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-glpcj pod in namespace openshift-image-registry (previous: false). W0424 07:14:47.269062 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 07:14:47.286383 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 07:14:47.286400 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-plk2p pod in namespace openshift-image-registry (previous: false). I0424 07:14:47.488442 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 07:14:47.488459 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-rsrn6 pod in namespace openshift-image-registry (previous: false). I0424 07:14:47.687621 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0424 07:14:47.687637 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-648d5b8c8c-phbs4 pod in namespace openshift-ingress (previous: false). I0424 07:14:47.888139 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-648d5b8c8c-phbs4 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-648d5b8c8c-phbs4\" is waiting to start: ContainerCreating" I0424 07:14:47.888153 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-648d5b8c8c-phbs4\" is waiting to start: ContainerCreating" I0424 07:14:47.888164 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6c86946764-fw9vz pod in namespace openshift-ingress (previous: false). I0424 07:14:48.087537 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6c86946764-fw9vz pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6c86946764-fw9vz\" is waiting to start: ContainerCreating" I0424 07:14:48.087552 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6c86946764-fw9vz\" is waiting to start: ContainerCreating" I0424 07:14:48.087562 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6c86946764-qpd25 pod in namespace openshift-ingress (previous: false). W0424 07:14:48.268881 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0424 07:14:48.293249 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6c86946764-qpd25 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6c86946764-qpd25\" is waiting to start: ContainerCreating" I0424 07:14:48.293262 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6c86946764-qpd25\" is waiting to start: ContainerCreating" I0424 07:14:48.293295 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dfv7w pod in namespace openshift-ingress-canary (previous: false). I0424 07:14:48.487869 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dfv7w pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dfv7w\" is waiting to start: ContainerCreating" I0424 07:14:48.487885 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dfv7w\" is waiting to start: ContainerCreating" I0424 07:14:48.487918 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-n2crf pod in namespace openshift-ingress-canary (previous: false). I0424 07:14:48.687440 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-n2crf pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-n2crf\" is waiting to start: ContainerCreating" I0424 07:14:48.687457 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-n2crf\" is waiting to start: ContainerCreating" I0424 07:14:48.687488 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-r9m5n pod in namespace openshift-ingress-canary (previous: false). I0424 07:14:48.886878 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-r9m5n pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-r9m5n\" is waiting to start: ContainerCreating" I0424 07:14:48.886893 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-r9m5n\" is waiting to start: ContainerCreating" I0424 07:14:48.886908 1 tasks_processing.go:74] worker 37 stopped. I0424 07:14:48.886990 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=0b0f8b401e8471aa3718f2c9c2220339170d012fa76c6a27c95d48dcf71f3f79 I0424 07:14:48.887041 1 recorder.go:75] Recording events/openshift-dns with fingerprint=416d54b5a0b794ac738ee45aabc4ea2afc05bd1898d5ec6e6ab333cb447ef2ee I0424 07:14:48.887142 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=0ec169e344758d4c92a918bc19f79335e229dc2ed8b7a9c09453666277753851 I0424 07:14:48.887169 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=f2ab229b9b05271961610b477a67d9932ac3593f6c6a61ea8d6f5a42cd51d6aa I0424 07:14:48.887214 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=cb300e64c2a06e4d550d7e12ac9fdef1b547d2cc7f5943c839272fa3a1906431 I0424 07:14:48.887230 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=34e707cbea6b700b68d4ebe9f65cbf96c33b11ba28e47f2ad54062738a076a26 I0424 07:14:48.887360 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-5dcrq with fingerprint=68b5a01ecff46b99f8674cf1793f6a47d4a803e80d21c94bf7b5ccc023759d09 I0424 07:14:48.887442 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-7dk97 with fingerprint=fb2237f84fe13f1ef0a40216d1d144556c268a5ab1b807e98aea5be0883c34a2 I0424 07:14:48.887513 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-965c7 with fingerprint=3f18917c684fe9c488eaf544b7edd72cd88cad584dcb42b4d6fa9346bde896dc I0424 07:14:48.887622 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-65d646d9c9-bgt6w with fingerprint=3472a3771deba37ae0f1effe40cf4a4725805db789272fe3b1b2b87b544d52f0 I0424 07:14:48.887711 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-65d646d9c9-dzxmk with fingerprint=16abb3388d729c4ac0c6f4fc428aed2e9ee3bb8ea90cfa0089ace88e07a6e92d I0424 07:14:48.887816 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-65fb668976-pxf98 with fingerprint=254ff58ff8a3f00ece45eb35afc7d9f21390200d659116a3115907512a98497d I0424 07:14:48.887873 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-dfv7w with fingerprint=c932f79042a4ea0902f11882b4696dff384fe56c4d39f039aee68928cd46d6dc I0424 07:14:48.887926 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-n2crf with fingerprint=172636e86f4a8614a063b1fbb8e01213cf3a98def6456eb2154a139be314d105 I0424 07:14:48.887983 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-r9m5n with fingerprint=15952b4b1554ce3ba680c33086da5a05cc6ccd7ff5ab1ca363a98bdb255a1533 I0424 07:14:48.887992 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.645825719s to process 15 records W0424 07:14:49.269713 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0424 07:14:49.269751 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0424 07:14:49.269788 1 tasks_processing.go:74] worker 13 stopped. E0424 07:14:49.269799 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0424 07:14:49.269811 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0424 07:14:49.269827 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0424 07:14:49.269838 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.028774693s to process 1 records I0424 07:14:56.676247 1 tasks_processing.go:74] worker 29 stopped. I0424 07:14:56.676287 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0424 07:14:56.676300 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.433479928s to process 1 records I0424 07:14:57.447672 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 07:14:57.453912 1 tasks_processing.go:74] worker 30 stopped. I0424 07:14:57.454179 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=b3a2c4fa45ed7ae58b9c4093be9a56682fdfea23d7350a26feed0cd7c1d1f420 I0424 07:14:57.454195 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.210487344s to process 1 records E0424 07:14:57.454249 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.213s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0424 07:14:57.455351 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0424 07:14:57.455364 1 periodic.go:209] Running workloads gatherer I0424 07:14:57.455381 1 tasks_processing.go:45] number of workers: 2 I0424 07:14:57.455392 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 07:14:57.455398 1 tasks_processing.go:71] worker 1 working on workload_info task. I0424 07:14:57.455460 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 07:14:57.455543 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0424 07:14:57.519949 1 tasks_processing.go:74] worker 0 stopped. I0424 07:14:57.519973 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 64.387286ms to process 0 records I0424 07:14:57.663423 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0424 07:14:57.671834 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (9ms) I0424 07:14:57.680865 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (9ms) I0424 07:14:57.689209 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0424 07:14:57.697080 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (8ms) I0424 07:14:57.706952 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (10ms) I0424 07:14:57.714954 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (8ms) I0424 07:14:57.722591 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (8ms) I0424 07:14:57.730880 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (8ms) I0424 07:14:57.738662 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (8ms) I0424 07:14:57.748482 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (10ms) I0424 07:14:57.772399 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (24ms) I0424 07:14:57.871312 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (99ms) I0424 07:14:57.972317 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (101ms) I0424 07:14:58.072315 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0424 07:14:58.172198 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0424 07:14:58.272182 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0424 07:14:58.374012 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (102ms) I0424 07:14:58.472396 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (98ms) I0424 07:14:58.573242 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (101ms) I0424 07:14:58.673342 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0424 07:14:58.771927 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (99ms) I0424 07:14:58.872315 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0424 07:14:58.972243 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0424 07:14:59.072894 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0424 07:14:59.172003 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (99ms) I0424 07:14:59.272450 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0424 07:14:59.371441 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (99ms) I0424 07:14:59.473507 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (102ms) I0424 07:14:59.571422 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (98ms) I0424 07:14:59.672269 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (101ms) I0424 07:14:59.774307 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (102ms) I0424 07:14:59.872959 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0424 07:14:59.952090 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 07:14:59.971735 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0424 07:14:59.971773 1 tasks_processing.go:74] worker 1 stopped. E0424 07:14:59.971783 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0424 07:14:59.972077 1 recorder.go:75] Recording config/workload_info with fingerprint=1fc06655b25ad045d3488baea6213e0cb9ccdbd3e1cdce2b13e510b221fbcc9f I0424 07:14:59.972092 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.516368479s to process 1 records E0424 07:14:59.972116 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.516s with: function \"workload_info\" failed with an error" I0424 07:14:59.973211 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0424 07:14:59.973221 1 periodic.go:209] Running conditional gatherer I0424 07:14:59.978787 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0424 07:14:59.985234 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.12:47674->172.30.0.10:53: read: connection refused E0424 07:14:59.985477 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0424 07:14:59.985541 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0424 07:14:59.990941 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0424 07:14:59.990955 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990960 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990964 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990968 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990970 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990973 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990976 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990979 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0424 07:14:59.990982 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0424 07:14:59.990996 1 tasks_processing.go:45] number of workers: 3 I0424 07:14:59.991005 1 tasks_processing.go:69] worker 2 listening for tasks. I0424 07:14:59.991009 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0424 07:14:59.991016 1 tasks_processing.go:69] worker 0 listening for tasks. I0424 07:14:59.991024 1 tasks_processing.go:69] worker 1 listening for tasks. I0424 07:14:59.991033 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0424 07:14:59.991035 1 tasks_processing.go:74] worker 1 stopped. I0424 07:14:59.991043 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0424 07:14:59.991088 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0424 07:14:59.991100 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 734ns to process 1 records I0424 07:14:59.991129 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0424 07:14:59.991137 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.056µs to process 1 records I0424 07:14:59.991142 1 tasks_processing.go:74] worker 0 stopped. I0424 07:14:59.991280 1 tasks_processing.go:74] worker 2 stopped. I0424 07:14:59.991291 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 229.437µs to process 0 records I0424 07:14:59.991308 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.12:47674->172.30.0.10:53: read: connection refused I0424 07:14:59.991325 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0424 07:15:00.012310 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=43b969f839b3e67c2ad27c1032aa7d30a9eef6e73dca0e24fdabbeff652192ba I0424 07:15:00.012426 1 diskrecorder.go:70] Writing 110 records to /var/lib/insights-operator/insights-2026-04-24-071500.tar.gz I0424 07:15:00.021084 1 diskrecorder.go:51] Wrote 110 records to disk in 8ms I0424 07:15:00.021126 1 periodic.go:278] Gathering cluster info every 2h0m0s I0424 07:15:00.021146 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0424 07:15:00.154520 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 07:15:11.214186 1 configmapobserver.go:84] configmaps "insights-config" not found I0424 07:16:08.755581 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="cdfa1af387c3d23daad6e8863f2b09e0732425f0c874c2b99208491d9569b5fe") W0424 07:16:08.755618 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0424 07:16:08.755669 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="011d4404834a1e325671cb3e85e108f3df8e0711c63b8891d24bde7abf72c63a") I0424 07:16:08.755718 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0424 07:16:08.755747 1 base_controller.go:181] Shutting down LoggingSyncer ... I0424 07:16:08.755748 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="701bc560e38ffdb63c662da8d2bf89e7337bfebd9a82afc6c916746fd251496c") I0424 07:16:08.755799 1 periodic.go:170] Shutting down I0424 07:16:08.755801 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0424 07:16:08.755819 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0424 07:16:08.755826 1 base_controller.go:113] All LoggingSyncer workers have been terminated I0424 07:16:08.755835 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0424 07:16:08.755838 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0424 07:16:08.755837 1 base_controller.go:181] Shutting down ConfigController ... I0424 07:16:08.755849 1 base_controller.go:113] All ConfigController workers have been terminated