W0423 19:32:46.400286 1 cmd.go:257] Using insecure, self-signed certificates I0423 19:32:46.649586 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:32:46.649907 1 observer_polling.go:159] Starting file observer I0423 19:32:47.148534 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0423 19:32:47.148717 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0423 19:32:47.149275 1 secure_serving.go:57] Forcing use of http/1.1 only W0423 19:32:47.149298 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0423 19:32:47.149302 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0423 19:32:47.149306 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0423 19:32:47.149309 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0423 19:32:47.149311 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0423 19:32:47.149314 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0423 19:32:47.149328 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0423 19:32:47.153202 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0423 19:32:47.153238 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"5ffc2afb-5616-4e70-a6ff-c3d4283112a0", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0423 19:32:47.154387 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0423 19:32:47.154427 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0423 19:32:47.154424 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0423 19:32:47.154443 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 19:32:47.155026 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0423 19:32:47.155043 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 19:32:47.155700 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3817199667/tls.crt::/tmp/serving-cert-3817199667/tls.key" I0423 19:32:47.155988 1 secure_serving.go:213] Serving securely on [::]:8443 I0423 19:32:47.156024 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0423 19:32:47.159098 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0423 19:32:47.159122 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 19:32:47.159242 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0423 19:32:47.164328 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 19:32:47.164349 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 19:32:47.168820 1 secretconfigobserver.go:119] support secret does not exist I0423 19:32:47.173366 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 19:32:47.177424 1 secretconfigobserver.go:119] support secret does not exist I0423 19:32:47.179692 1 recorder.go:176] Pruning old reports every 4h39m49s, max age is 288h0m0s I0423 19:32:47.184640 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0423 19:32:47.184645 1 periodic.go:216] Running clusterconfig gatherer I0423 19:32:47.184663 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0423 19:32:47.184684 1 tasks_processing.go:45] number of workers: 64 I0423 19:32:47.184701 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0423 19:32:47.184711 1 insightsreport.go:296] Starting report retriever I0423 19:32:47.184718 1 tasks_processing.go:69] worker 4 listening for tasks. I0423 19:32:47.184720 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0423 19:32:47.184726 1 tasks_processing.go:69] worker 7 listening for tasks. I0423 19:32:47.184733 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 19:32:47.184734 1 tasks_processing.go:69] worker 5 listening for tasks. I0423 19:32:47.184737 1 tasks_processing.go:69] worker 6 listening for tasks. I0423 19:32:47.184745 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 19:32:47.184748 1 tasks_processing.go:69] worker 9 listening for tasks. I0423 19:32:47.184740 1 tasks_processing.go:69] worker 8 listening for tasks. I0423 19:32:47.184751 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 19:32:47.184762 1 tasks_processing.go:69] worker 3 listening for tasks. I0423 19:32:47.184820 1 tasks_processing.go:69] worker 19 listening for tasks. I0423 19:32:47.184822 1 tasks_processing.go:69] worker 27 listening for tasks. I0423 19:32:47.184824 1 tasks_processing.go:69] worker 24 listening for tasks. I0423 19:32:47.184832 1 tasks_processing.go:69] worker 25 listening for tasks. I0423 19:32:47.184836 1 tasks_processing.go:69] worker 10 listening for tasks. I0423 19:32:47.184840 1 tasks_processing.go:69] worker 15 listening for tasks. I0423 19:32:47.184843 1 tasks_processing.go:69] worker 34 listening for tasks. I0423 19:32:47.184841 1 tasks_processing.go:69] worker 20 listening for tasks. I0423 19:32:47.184852 1 tasks_processing.go:69] worker 26 listening for tasks. I0423 19:32:47.184856 1 tasks_processing.go:69] worker 13 listening for tasks. I0423 19:32:47.184857 1 tasks_processing.go:69] worker 12 listening for tasks. I0423 19:32:47.184847 1 tasks_processing.go:69] worker 14 listening for tasks. I0423 19:32:47.184860 1 tasks_processing.go:69] worker 23 listening for tasks. I0423 19:32:47.184869 1 tasks_processing.go:69] worker 18 listening for tasks. I0423 19:32:47.184859 1 tasks_processing.go:69] worker 16 listening for tasks. I0423 19:32:47.184848 1 tasks_processing.go:69] worker 21 listening for tasks. I0423 19:32:47.184875 1 tasks_processing.go:69] worker 17 listening for tasks. I0423 19:32:47.184879 1 tasks_processing.go:69] worker 30 listening for tasks. I0423 19:32:47.184881 1 tasks_processing.go:69] worker 41 listening for tasks. I0423 19:32:47.184882 1 tasks_processing.go:69] worker 33 listening for tasks. I0423 19:32:47.184881 1 tasks_processing.go:69] worker 38 listening for tasks. I0423 19:32:47.184862 1 tasks_processing.go:69] worker 28 listening for tasks. I0423 19:32:47.184889 1 tasks_processing.go:69] worker 39 listening for tasks. I0423 19:32:47.184890 1 tasks_processing.go:69] worker 37 listening for tasks. I0423 19:32:47.184894 1 tasks_processing.go:69] worker 35 listening for tasks. I0423 19:32:47.184867 1 tasks_processing.go:69] worker 29 listening for tasks. I0423 19:32:47.184871 1 tasks_processing.go:69] worker 11 listening for tasks. I0423 19:32:47.184873 1 tasks_processing.go:69] worker 31 listening for tasks. I0423 19:32:47.184901 1 tasks_processing.go:69] worker 36 listening for tasks. I0423 19:32:47.184874 1 tasks_processing.go:69] worker 32 listening for tasks. I0423 19:32:47.184881 1 tasks_processing.go:69] worker 40 listening for tasks. I0423 19:32:47.184852 1 tasks_processing.go:69] worker 22 listening for tasks. I0423 19:32:47.184956 1 tasks_processing.go:69] worker 53 listening for tasks. I0423 19:32:47.184958 1 tasks_processing.go:69] worker 49 listening for tasks. I0423 19:32:47.184962 1 tasks_processing.go:69] worker 42 listening for tasks. I0423 19:32:47.184972 1 tasks_processing.go:69] worker 51 listening for tasks. I0423 19:32:47.184964 1 tasks_processing.go:69] worker 55 listening for tasks. I0423 19:32:47.184963 1 tasks_processing.go:69] worker 56 listening for tasks. I0423 19:32:47.184973 1 tasks_processing.go:69] worker 54 listening for tasks. I0423 19:32:47.184984 1 tasks_processing.go:69] worker 46 listening for tasks. I0423 19:32:47.184984 1 tasks_processing.go:69] worker 57 listening for tasks. I0423 19:32:47.184989 1 tasks_processing.go:69] worker 52 listening for tasks. I0423 19:32:47.184994 1 tasks_processing.go:69] worker 44 listening for tasks. I0423 19:32:47.184995 1 tasks_processing.go:69] worker 48 listening for tasks. I0423 19:32:47.184998 1 tasks_processing.go:69] worker 43 listening for tasks. I0423 19:32:47.185001 1 tasks_processing.go:69] worker 60 listening for tasks. I0423 19:32:47.185003 1 tasks_processing.go:69] worker 45 listening for tasks. I0423 19:32:47.185005 1 tasks_processing.go:71] worker 48 working on dvo_metrics task. I0423 19:32:47.184993 1 tasks_processing.go:69] worker 62 listening for tasks. I0423 19:32:47.185010 1 tasks_processing.go:71] worker 45 working on machine_configs task. I0423 19:32:47.185013 1 tasks_processing.go:71] worker 7 working on operators_pods_and_events task. I0423 19:32:47.185014 1 tasks_processing.go:71] worker 25 working on container_runtime_configs task. I0423 19:32:47.185020 1 tasks_processing.go:71] worker 35 working on metrics task. I0423 19:32:47.185023 1 tasks_processing.go:71] worker 0 working on openstack_dataplanedeployments task. I0423 19:32:47.185026 1 tasks_processing.go:71] worker 30 working on qemu_kubevirt_launcher_logs task. I0423 19:32:47.185029 1 tasks_processing.go:71] worker 38 working on cost_management_metrics_configs task. I0423 19:32:47.185031 1 tasks_processing.go:71] worker 8 working on networks task. I0423 19:32:47.185036 1 tasks_processing.go:71] worker 5 working on pod_network_connectivity_checks task. I0423 19:32:47.185050 1 tasks_processing.go:71] worker 55 working on jaegers task. I0423 19:32:47.185071 1 tasks_processing.go:71] worker 36 working on pdbs task. I0423 19:32:47.185076 1 tasks_processing.go:71] worker 11 working on silenced_alerts task. I0423 19:32:47.185080 1 tasks_processing.go:71] worker 4 working on image_registries task. I0423 19:32:47.185118 1 tasks_processing.go:71] worker 29 working on cluster_apiserver task. I0423 19:32:47.185326 1 tasks_processing.go:71] worker 51 working on support_secret task. I0423 19:32:47.185395 1 tasks_processing.go:71] worker 12 working on node_logs task. I0423 19:32:47.185494 1 tasks_processing.go:71] worker 34 working on nodes task. I0423 19:32:47.185545 1 tasks_processing.go:71] worker 46 working on openstack_dataplanenodesets task. I0423 19:32:47.185550 1 tasks_processing.go:71] worker 40 working on node_features task. I0423 19:32:47.185015 1 tasks_processing.go:69] worker 63 listening for tasks. I0423 19:32:47.185618 1 tasks_processing.go:71] worker 56 working on nodenetworkconfigurationpolicies task. I0423 19:32:47.185022 1 tasks_processing.go:71] worker 33 working on sap_config task. I0423 19:32:47.185760 1 tasks_processing.go:71] worker 23 working on olm_operators task. I0423 19:32:47.185530 1 tasks_processing.go:71] worker 19 working on machine_autoscalers task. I0423 19:32:47.186160 1 tasks_processing.go:71] worker 52 working on aggregated_monitoring_cr_names task. I0423 19:32:47.185422 1 tasks_processing.go:71] worker 15 working on storage_classes task. I0423 19:32:47.186309 1 tasks_processing.go:71] worker 16 working on container_images task. I0423 19:32:47.185026 1 tasks_processing.go:71] worker 9 working on machine_healthchecks task. I0423 19:32:47.186413 1 tasks_processing.go:71] worker 44 working on operators task. I0423 19:32:47.186436 1 tasks_processing.go:71] worker 57 working on ceph_cluster task. I0423 19:32:47.186490 1 tasks_processing.go:71] worker 26 working on active_alerts task. I0423 19:32:47.186534 1 tasks_processing.go:71] worker 21 working on openstack_controlplanes task. I0423 19:32:47.185119 1 tasks_processing.go:71] worker 53 working on openshift_machine_api_events task. W0423 19:32:47.185056 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:32:47.186849 1 tasks_processing.go:71] worker 35 working on sap_datahubs task. I0423 19:32:47.185383 1 tasks_processing.go:71] worker 2 working on version task. I0423 19:32:47.185013 1 tasks_processing.go:71] worker 1 working on validating_webhook_configurations task. W0423 19:32:47.185653 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory W0423 19:32:47.186554 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:32:47.185005 1 tasks_processing.go:71] worker 43 working on sap_pods task. I0423 19:32:47.187119 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 1.779803ms to process 0 records I0423 19:32:47.187158 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.96199ms to process 0 records I0423 19:32:47.187170 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 563.707µs to process 0 records I0423 19:32:47.187261 1 tasks_processing.go:71] worker 11 working on ingress_certificates task. I0423 19:32:47.187516 1 tasks_processing.go:71] worker 22 working on image task. I0423 19:32:47.185117 1 tasks_processing.go:71] worker 13 working on config_maps task. I0423 19:32:47.187661 1 tasks_processing.go:71] worker 26 working on clusterroles task. I0423 19:32:47.185007 1 tasks_processing.go:71] worker 6 working on storage_cluster task. I0423 19:32:47.185006 1 tasks_processing.go:69] worker 61 listening for tasks. I0423 19:32:47.184999 1 tasks_processing.go:69] worker 47 listening for tasks. I0423 19:32:47.187845 1 tasks_processing.go:71] worker 61 working on machine_sets task. I0423 19:32:47.185008 1 tasks_processing.go:71] worker 27 working on crds task. I0423 19:32:47.185624 1 tasks_processing.go:71] worker 63 working on schedulers task. I0423 19:32:47.185009 1 tasks_processing.go:71] worker 32 working on nodenetworkstates task. I0423 19:32:47.186523 1 tasks_processing.go:71] worker 18 working on lokistack task. I0423 19:32:47.185008 1 tasks_processing.go:71] worker 24 working on openstack_version task. I0423 19:32:47.185014 1 tasks_processing.go:71] worker 37 working on feature_gates task. I0423 19:32:47.184986 1 tasks_processing.go:69] worker 50 listening for tasks. I0423 19:32:47.185014 1 tasks_processing.go:71] worker 39 working on certificate_signing_requests task. I0423 19:32:47.185022 1 tasks_processing.go:71] worker 14 working on tsdb_status task. I0423 19:32:47.185067 1 tasks_processing.go:71] worker 31 working on machines task. I0423 19:32:47.185118 1 tasks_processing.go:71] worker 41 working on infrastructures task. I0423 19:32:47.185122 1 tasks_processing.go:71] worker 28 working on install_plans task. I0423 19:32:47.185000 1 tasks_processing.go:69] worker 58 listening for tasks. I0423 19:32:47.185405 1 tasks_processing.go:71] worker 49 working on authentication task. I0423 19:32:47.185016 1 tasks_processing.go:71] worker 62 working on machine_config_pools task. I0423 19:32:47.185325 1 tasks_processing.go:71] worker 10 working on service_accounts task. I0423 19:32:47.185536 1 tasks_processing.go:71] worker 3 working on mutating_webhook_configurations task. I0423 19:32:47.185108 1 tasks_processing.go:71] worker 60 working on image_pruners task. I0423 19:32:47.185652 1 tasks_processing.go:71] worker 20 working on ingress task. I0423 19:32:47.185003 1 tasks_processing.go:71] worker 17 working on oauths task. I0423 19:32:47.185006 1 tasks_processing.go:69] worker 59 listening for tasks. I0423 19:32:47.188539 1 tasks_processing.go:74] worker 59 stopped. I0423 19:32:47.185642 1 tasks_processing.go:71] worker 54 working on proxies task. I0423 19:32:47.188588 1 tasks_processing.go:71] worker 42 working on openshift_logging task. W0423 19:32:47.188962 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:32:47.188977 1 tasks_processing.go:74] worker 14 stopped. I0423 19:32:47.188987 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 1.049348ms to process 0 records I0423 19:32:47.189157 1 tasks_processing.go:71] worker 58 working on overlapping_namespace_uids task. I0423 19:32:47.189174 1 tasks_processing.go:71] worker 47 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0423 19:32:47.189201 1 tasks_processing.go:74] worker 38 stopped. I0423 19:32:47.189201 1 tasks_processing.go:71] worker 50 working on monitoring_persistent_volumes task. I0423 19:32:47.189230 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 4.161143ms to process 0 records I0423 19:32:47.189487 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0423 19:32:47.189504 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 19:32:47.189510 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0423 19:32:47.189513 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0423 19:32:47.189528 1 controller.go:531] The operator is still being initialized I0423 19:32:47.189533 1 controller.go:554] The operator is healthy I0423 19:32:47.192151 1 tasks_processing.go:74] worker 56 stopped. I0423 19:32:47.192166 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 6.438071ms to process 0 records I0423 19:32:47.200116 1 tasks_processing.go:74] worker 33 stopped. I0423 19:32:47.200125 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 14.379686ms to process 0 records I0423 19:32:47.200154 1 tasks_processing.go:74] worker 0 stopped. I0423 19:32:47.200169 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 15.122416ms to process 0 records I0423 19:32:47.200185 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 15.132826ms to process 0 records I0423 19:32:47.200196 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 14.028426ms to process 0 records I0423 19:32:47.200201 1 tasks_processing.go:74] worker 25 stopped. I0423 19:32:47.200204 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 13.302115ms to process 0 records I0423 19:32:47.200210 1 tasks_processing.go:74] worker 35 stopped. I0423 19:32:47.200216 1 tasks_processing.go:74] worker 19 stopped. E0423 19:32:47.200218 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0423 19:32:47.200228 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 15.130403ms to process 0 records I0423 19:32:47.200237 1 tasks_processing.go:74] worker 5 stopped. I0423 19:32:47.200303 1 tasks_processing.go:74] worker 9 stopped. E0423 19:32:47.200316 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0423 19:32:47.200326 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 13.945493ms to process 0 records E0423 19:32:47.200396 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0423 19:32:47.200404 1 tasks_processing.go:74] worker 40 stopped. I0423 19:32:47.200410 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 14.838862ms to process 0 records I0423 19:32:47.200443 1 tasks_processing.go:74] worker 46 stopped. I0423 19:32:47.200455 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 14.880672ms to process 0 records I0423 19:32:47.200718 1 tasks_processing.go:74] worker 36 stopped. I0423 19:32:47.200870 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=1df3a04e7e56db58fa5e98804aa1590fe47f2beea1b491701369cde0cf29830c I0423 19:32:47.200891 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=440db68496bf59fd663fe2070b523fbbcaffc66a1a8786419a8d43e5d0632f80 I0423 19:32:47.200906 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=8917e6d326e2976964b34d5f6e06924038e0c6bff478627ebb290f7f57be4d90 I0423 19:32:47.200914 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 15.63767ms to process 3 records I0423 19:32:47.201026 1 tasks_processing.go:74] worker 8 stopped. I0423 19:32:47.201055 1 recorder.go:75] Recording config/network with fingerprint=c1764bde72c41a940e5a7b5e2a070d926edbfb893d7e9841100e7d9e114acded I0423 19:32:47.201064 1 gather.go:177] gatherer "clusterconfig" function "networks" took 15.698434ms to process 1 records I0423 19:32:47.201070 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 15.325399ms to process 0 records I0423 19:32:47.201109 1 tasks_processing.go:74] worker 12 stopped. I0423 19:32:47.201121 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=b84d3513a8b936ed9bb735fd2b2b0e9b06a81cc488ad99e2186441ecfe269b38 I0423 19:32:47.201132 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=dbe8e7ff94c5196b0b2b8edfbc1dbf5c94cf4b7135c2a3f684b08693bb3937f6 I0423 19:32:47.201155 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 14.583541ms to process 2 records I0423 19:32:47.201154 1 tasks_processing.go:74] worker 15 stopped. I0423 19:32:47.201927 1 gather_logs.go:145] no pods in namespace were found I0423 19:32:47.201942 1 tasks_processing.go:74] worker 30 stopped. I0423 19:32:47.201948 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 16.904883ms to process 0 records I0423 19:32:47.202837 1 tasks_processing.go:74] worker 18 stopped. I0423 19:32:47.202854 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 14.928308ms to process 0 records I0423 19:32:47.203055 1 tasks_processing.go:74] worker 22 stopped. I0423 19:32:47.203184 1 recorder.go:75] Recording config/image with fingerprint=57fb4887d3f018d8bd6449bfa4f12f18ec5b4c084609665efbcbdde2e967c289 I0423 19:32:47.203202 1 gather.go:177] gatherer "clusterconfig" function "image" took 15.520244ms to process 1 records I0423 19:32:47.203216 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 15.322538ms to process 0 records I0423 19:32:47.203227 1 tasks_processing.go:74] worker 6 stopped. W0423 19:32:47.204506 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:32:47.222577 1 tasks_processing.go:74] worker 62 stopped. I0423 19:32:47.222591 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 34.623114ms to process 0 records I0423 19:32:47.222597 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 34.692659ms to process 0 records I0423 19:32:47.222602 1 tasks_processing.go:74] worker 32 stopped. I0423 19:32:47.222746 1 tasks_processing.go:74] worker 31 stopped. E0423 19:32:47.222760 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0423 19:32:47.222773 1 gather.go:177] gatherer "clusterconfig" function "machines" took 34.813044ms to process 0 records I0423 19:32:47.222983 1 tasks_processing.go:74] worker 17 stopped. I0423 19:32:47.223224 1 recorder.go:75] Recording config/oauth with fingerprint=d6c5291bd136ccbd0c6195837aea25a4eef2789e3651b37e8c0198a2ffe7229d I0423 19:32:47.223238 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 34.411903ms to process 1 records I0423 19:32:47.223325 1 tasks_processing.go:74] worker 60 stopped. I0423 19:32:47.223414 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=33c1fb6867787376fef1b304be2dfaa9f6c771c24fae3a274a3b49221967d2d6 I0423 19:32:47.223636 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 34.90892ms to process 1 records I0423 19:32:47.223743 1 tasks_processing.go:74] worker 63 stopped. I0423 19:32:47.223805 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=d66847f2a37d837d828bfb9a03f72f6ba30e91be50f2c1bed65191f066273406 I0423 19:32:47.223829 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 35.127061ms to process 1 records I0423 19:32:47.223932 1 tasks_processing.go:74] worker 49 stopped. I0423 19:32:47.224251 1 recorder.go:75] Recording config/authentication with fingerprint=6976351472632268264673243746b9941c4e416fcfecb2162be95c0bc2c9906b I0423 19:32:47.224316 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 35.15413ms to process 1 records I0423 19:32:47.227639 1 tasks_processing.go:74] worker 57 stopped. I0423 19:32:47.227649 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 41.1921ms to process 0 records I0423 19:32:47.227664 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 41.055337ms to process 0 records I0423 19:32:47.227668 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 39.739754ms to process 0 records I0423 19:32:47.227672 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 40.867669ms to process 0 records I0423 19:32:47.227677 1 tasks_processing.go:74] worker 53 stopped. I0423 19:32:47.227681 1 tasks_processing.go:74] worker 21 stopped. I0423 19:32:47.227685 1 tasks_processing.go:74] worker 24 stopped. I0423 19:32:47.227815 1 tasks_processing.go:74] worker 29 stopped. I0423 19:32:47.227939 1 recorder.go:75] Recording config/apiserver with fingerprint=9ddfa634ec8148ef967a9323eef4167f0def9ff7af22791b42ec54038d2e296a I0423 19:32:47.227954 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 42.470348ms to process 1 records I0423 19:32:47.228148 1 tasks_processing.go:74] worker 41 stopped. I0423 19:32:47.228409 1 recorder.go:75] Recording config/infrastructure with fingerprint=746e4a68d522f1d48991505010bfe6c4dfb224e7fe56b1e1c805e1c0d81b9046 I0423 19:32:47.228420 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 39.964285ms to process 1 records E0423 19:32:47.228426 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0423 19:32:47.228431 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 42.942072ms to process 0 records I0423 19:32:47.228436 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 41.232959ms to process 0 records I0423 19:32:47.228440 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 39.745952ms to process 0 records I0423 19:32:47.228446 1 tasks_processing.go:74] worker 42 stopped. I0423 19:32:47.228450 1 tasks_processing.go:74] worker 43 stopped. I0423 19:32:47.228449 1 tasks_processing.go:74] worker 51 stopped. I0423 19:32:47.228525 1 tasks_processing.go:74] worker 54 stopped. I0423 19:32:47.228622 1 recorder.go:75] Recording config/proxy with fingerprint=7805578463b208b01d3b58286af4791c9a03eb0e5f78a36b30ee86e6d7c5877a I0423 19:32:47.228640 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 39.969425ms to process 1 records I0423 19:32:47.228735 1 tasks_processing.go:74] worker 20 stopped. I0423 19:32:47.228783 1 recorder.go:75] Recording config/ingress with fingerprint=6dd77e24064fa1c813f8e9690d0b2ac79d07eae5610f87a9dc116b7323dfa2cf I0423 19:32:47.228797 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 40.518917ms to process 1 records I0423 19:32:47.228831 1 tasks_processing.go:74] worker 4 stopped. I0423 19:32:47.229125 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=fd616bdcc2288f967db14e14db0b5bd64d9c3b3be6d7b15f271ade54ae9dfe5b I0423 19:32:47.229153 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 43.728664ms to process 1 records I0423 19:32:47.229498 1 tasks_processing.go:74] worker 3 stopped. I0423 19:32:47.229714 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=ddf09ec4ae2265ad3b7e585123b508df3404ab0f3bc893c01cf74ec2dd8de374 I0423 19:32:47.229748 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=02c59e3343a85d2eee7083bbbabad2831b813dde8e707eae67b608fb9a5b8ae9 I0423 19:32:47.229772 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=58d351a8bb97c49f9b0c923bdb7f5377db57491639b66eaa24b11b6489e4e30d I0423 19:32:47.229781 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 41.535434ms to process 3 records I0423 19:32:47.229793 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 41.866089ms to process 0 records I0423 19:32:47.229799 1 tasks_processing.go:74] worker 61 stopped. I0423 19:32:47.229866 1 tasks_processing.go:74] worker 52 stopped. I0423 19:32:47.229880 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 43.686723ms to process 0 records I0423 19:32:47.230235 1 tasks_processing.go:74] worker 55 stopped. I0423 19:32:47.230246 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 45.160079ms to process 0 records I0423 19:32:47.230545 1 tasks_processing.go:74] worker 45 stopped. E0423 19:32:47.230556 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0423 19:32:47.230576 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0423 19:32:47.230587 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 45.527864ms to process 1 records I0423 19:32:47.230617 1 tasks_processing.go:74] worker 34 stopped. I0423 19:32:47.230981 1 recorder.go:75] Recording config/node/ip-10-0-0-245.ec2.internal with fingerprint=20c16006ccc86b67735e7705f02759d479c9c2334aac50a994131a24f50830e9 I0423 19:32:47.231066 1 recorder.go:75] Recording config/node/ip-10-0-1-108.ec2.internal with fingerprint=43dd9e185b5580e29f06ebd844591d9d9eeaf2ebe18234112007fee60ece3819 I0423 19:32:47.231159 1 recorder.go:75] Recording config/node/ip-10-0-2-122.ec2.internal with fingerprint=7dd9a86a31acf6910415baa4b658c0249d0d601162e445bca86743aa90a5df34 I0423 19:32:47.231169 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 45.064161ms to process 3 records I0423 19:32:47.231180 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 41.426128ms to process 0 records I0423 19:32:47.231198 1 tasks_processing.go:74] worker 50 stopped. I0423 19:32:47.231270 1 tasks_processing.go:74] worker 37 stopped. I0423 19:32:47.231281 1 recorder.go:75] Recording config/featuregate with fingerprint=4700588790e8bfaf9ff815493bb5e820a9b2153409049d15daf30922aed65e06 I0423 19:32:47.231288 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 42.920993ms to process 1 records I0423 19:32:47.231379 1 tasks_processing.go:74] worker 1 stopped. I0423 19:32:47.231418 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=a79d114e66ddcf36fdc008b49b8e867cb10fe595b2f050ccf75df88afb33c8d4 I0423 19:32:47.231497 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=b6b584aaaf186f0fd6608f574155506527fe4c2f2d00790f8607cac5f41bc202 I0423 19:32:47.231516 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=c8b4ea6c8d332b630ca639ec6242c25f2ec40dbd9c595bd3b7133c677f776862 I0423 19:32:47.231538 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=6db704a6b687e418ccc38e10d880f3b2643425b447b1181375c0bebff928a177 I0423 19:32:47.231561 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=25d69695629fc0793a2104cfd9b6ba77e62e23f4651c4562b211ee6f29bf367b I0423 19:32:47.231589 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=d8c5f12428d420ec7c78435bd77b6cc888e970e0a3bebffd09c5c9a4e0b2c894 I0423 19:32:47.231628 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=166097eb62314fb0b9ac55340f050cf4ac56c1364563fb0a0702bad75cc701ca I0423 19:32:47.231660 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=32e3367a2fe15cdba4b74426eb4a54d46ed5045e1ecd6fdb464de9f9b96c3c10 I0423 19:32:47.231683 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=97b2d8640b55be8ef73b43fe164092661a10d3b6818ddbd5b32d7f07f65ad677 I0423 19:32:47.231706 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=9fa0f74ad419465b7ee8b4a2f3e69d0e0d0e7664b3effb16d409b53a20b20a22 I0423 19:32:47.231728 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=0a878d3e4668173e958bf094567c3a2c5f0bd6bbbf8e35134fd0269552a366e9 I0423 19:32:47.231734 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 44.02449ms to process 11 records I0423 19:32:47.235279 1 tasks_processing.go:74] worker 58 stopped. I0423 19:32:47.235311 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0423 19:32:47.235323 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 46.099143ms to process 1 records I0423 19:32:47.235373 1 tasks_processing.go:74] worker 39 stopped. I0423 19:32:47.235388 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 47.446404ms to process 0 records I0423 19:32:47.238787 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0423 19:32:47.238804 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0423 19:32:47.238853 1 tasks_processing.go:74] worker 23 stopped. I0423 19:32:47.238915 1 recorder.go:75] Recording config/olm_operators with fingerprint=ee8cb101342ce18f754421fbfa1f52b4a0e4dc7b75dc1932f4edf25a91141368 I0423 19:32:47.238931 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 53.074238ms to process 1 records W0423 19:32:47.238959 1 operator.go:328] started I0423 19:32:47.238992 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0423 19:32:47.239834 1 tasks_processing.go:74] worker 26 stopped. I0423 19:32:47.240000 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=8130f9d17aa5a6ae3a4f214fe4f2e97aff7d91247e1ba94dfa78a9ab8ced1a9d I0423 19:32:47.240086 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=c3a96a8bfa25b2f0dcaa28baaee6151183360565931799c80d6abc2614f9416b I0423 19:32:47.240095 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 52.15289ms to process 2 records I0423 19:32:47.248351 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0423 19:32:47.248363 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0423 19:32:47.248368 1 controller.go:254] Source scaController *sca.Controller is not ready I0423 19:32:47.248371 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0423 19:32:47.248374 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 19:32:47.248389 1 controller.go:531] The operator is still being initialized I0423 19:32:47.248394 1 controller.go:554] The operator is healthy I0423 19:32:47.253628 1 prometheus_rules.go:88] Prometheus rules successfully created I0423 19:32:47.254539 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0423 19:32:47.254551 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 19:32:47.255357 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file E0423 19:32:47.257043 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2741dacb30-2965-4cf3-84c3-e11817523de2%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.16:56230->172.30.0.10:53: read: connection refused I0423 19:32:47.257049 1 tasks_processing.go:74] worker 27 stopped. I0423 19:32:47.257063 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2741dacb30-2965-4cf3-84c3-e11817523de2%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.16:56230->172.30.0.10:53: read: connection refused I0423 19:32:47.257811 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=2041a5cec7ff9f98e28f2c9c0a6c2e1b76e40503cdac7eb1817a31f3e4403539 I0423 19:32:47.258188 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=1886081468c769e23b82079404a603b7cf47a42c578c1412e2f3fab764810139 I0423 19:32:47.258207 1 gather.go:177] gatherer "clusterconfig" function "crds" took 68.840767ms to process 2 records I0423 19:32:47.258734 1 tasks_processing.go:74] worker 16 stopped. I0423 19:32:47.258812 1 recorder.go:75] Recording config/running_containers with fingerprint=ce131d72c8f311f2f9d3a0b9e04f0f6a0e58e48f978466e14732c57cba851c3e I0423 19:32:47.258825 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 72.400442ms to process 1 records I0423 19:32:47.259871 1 base_controller.go:82] Caches are synced for ConfigController I0423 19:32:47.259881 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0423 19:32:47.261362 1 tasks_processing.go:74] worker 47 stopped. I0423 19:32:47.261373 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 72.172403ms to process 0 records I0423 19:32:47.262832 1 tasks_processing.go:74] worker 13 stopped. E0423 19:32:47.262846 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0423 19:32:47.262851 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0423 19:32:47.262855 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0423 19:32:47.262863 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=ad286723f58bdcfc37aeba3ec5b4110c08e1af59cd34d14b4bfaab02d18e4856 I0423 19:32:47.262902 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0423 19:32:47.262910 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=2ab5ab7b1b10d7fcf1197bb24dea7c90f400e4effc18c7356873209d54fdf84b I0423 19:32:47.262914 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0423 19:32:47.262918 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0423 19:32:47.262955 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0423 19:32:47.262962 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0423 19:32:47.262967 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 75.233215ms to process 7 records I0423 19:32:47.265606 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:32:47.301942 1 tasks_processing.go:74] worker 2 stopped. I0423 19:32:47.302185 1 recorder.go:75] Recording config/version with fingerprint=8eaa7ac5742662205e525e8885dfee561c82a85785f3d7d1cc7e31592d2d5226 I0423 19:32:47.302202 1 recorder.go:75] Recording config/id with fingerprint=48818c20de212943014e0ed45631f52bcd54fea2b0b430a98b0d00c7ef840b57 I0423 19:32:47.302209 1 gather.go:177] gatherer "clusterconfig" function "version" took 114.980761ms to process 2 records I0423 19:32:47.309167 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0423 19:32:47.312711 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.16:54264->172.30.0.10:53: read: connection refused I0423 19:32:47.312723 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.16:54264->172.30.0.10:53: read: connection refused I0423 19:32:47.339034 1 base_controller.go:82] Caches are synced for LoggingSyncer I0423 19:32:47.339050 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0423 19:32:47.341997 1 tasks_processing.go:74] worker 11 stopped. E0423 19:32:47.342013 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0423 19:32:47.342019 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ps23krgrug0inkj9hc8sh6m5b2n43rp-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ps23krgrug0inkj9hc8sh6m5b2n43rp-primary-cert-bundle-secret" not found I0423 19:32:47.342080 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=2d6fd092587fffebe72d8b8dd23081298d13d0cffd52be0a49d58ac8b160e732 I0423 19:32:47.342094 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 154.717628ms to process 1 records I0423 19:32:47.659721 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0423 19:32:47.659737 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0423 19:32:47.659927 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-ln9x5 pod in namespace openshift-dns (previous: false). I0423 19:32:47.890257 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-ln9x5 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-ln9x5\" is waiting to start: ContainerCreating" I0423 19:32:47.890278 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-ln9x5\" is waiting to start: ContainerCreating" I0423 19:32:47.890288 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-ln9x5 pod in namespace openshift-dns (previous: false). I0423 19:32:48.061669 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-ln9x5 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-ln9x5\" is waiting to start: ContainerCreating" I0423 19:32:48.061694 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-ln9x5\" is waiting to start: ContainerCreating" I0423 19:32:48.061722 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-t9t42 pod in namespace openshift-dns (previous: false). W0423 19:32:48.204599 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:32:48.248966 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-t9t42 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-t9t42\" is waiting to start: ContainerCreating" I0423 19:32:48.248982 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-t9t42\" is waiting to start: ContainerCreating" I0423 19:32:48.248990 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-t9t42 pod in namespace openshift-dns (previous: false). I0423 19:32:48.445880 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-t9t42 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-t9t42\" is waiting to start: ContainerCreating" I0423 19:32:48.445896 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-t9t42\" is waiting to start: ContainerCreating" I0423 19:32:48.445929 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-wghf4 pod in namespace openshift-dns (previous: false). I0423 19:32:48.650824 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0423 19:32:48.666583 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wghf4 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-wghf4\" is waiting to start: ContainerCreating" I0423 19:32:48.666596 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-wghf4\" is waiting to start: ContainerCreating" I0423 19:32:48.666603 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-wghf4 pod in namespace openshift-dns (previous: false). I0423 19:32:48.846098 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wghf4 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-wghf4\" is waiting to start: ContainerCreating" I0423 19:32:48.846116 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-wghf4\" is waiting to start: ContainerCreating" I0423 19:32:48.846128 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-895k6 pod in namespace openshift-dns (previous: false). I0423 19:32:49.047585 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:32:49.047602 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-jhhsl pod in namespace openshift-dns (previous: false). W0423 19:32:49.204334 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:32:49.249283 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:32:49.249298 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-mrv68 pod in namespace openshift-dns (previous: false). I0423 19:32:49.254280 1 tasks_processing.go:74] worker 44 stopped. I0423 19:32:49.254326 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=8b27e0b57c75f7bcfef8783698631f8c90d3ee3c0d65df00e41afea3b790c231 I0423 19:32:49.254370 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=469e8d1c317755f5557f79fbc75adb19139524a7e441d219e83441ac9c59dd43 I0423 19:32:49.254399 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0423 19:32:49.254424 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=b0892b10a527cd08ebac1fa460627ac0dedc2247da4d91a6848887ca3b65647e I0423 19:32:49.254441 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0423 19:32:49.254478 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=714bc99ad15f7cdc06bf019c76393cd7c11f60f32aa1c49a24d4f71071132587 I0423 19:32:49.254508 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=3326a8ab7e7d91997dd014367882557e437f513914095a76d9a630479f1556d1 I0423 19:32:49.254555 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=80574364b5c49db8fd2c34a2766ec6d4e0283f23ed132d07232d185a1542ed2f I0423 19:32:49.254569 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=d8cbc62e03ad2e7b1e1aceb902a25c37a4c42cfc4b53f9769e07963f6e9147aa I0423 19:32:49.254586 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=ca673e18ce8c22b407d67f87e7f66fe61f376963c482357c7709a948b85dc2d2 I0423 19:32:49.254595 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0423 19:32:49.254611 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=d5ad985a0863ec34e011df769aaa067748c0c0fd5182da01a43ea7b5a1073645 I0423 19:32:49.254619 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0423 19:32:49.254635 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=8035af64818380621b46754422f0841c5076bb18a6298ca44798da4c7868679d I0423 19:32:49.254644 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0423 19:32:49.254656 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=3e6f859902dbf50bf4dc7fdee0c1f0aa332b5828dc31c813c054be02be5adc2c I0423 19:32:49.254663 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0423 19:32:49.254677 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=306c7e3ebcade01492c5607f2d6464e7da80fcaa5b7e5b8eddc5034a1d6f00c8 I0423 19:32:49.254801 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=facb39c5fdb674c066a15a838aa67d21af218d09489a3fa5b91b827d6d90b212 I0423 19:32:49.254811 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0423 19:32:49.254819 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0423 19:32:49.254840 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0423 19:32:49.254863 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=c8edae2dc965d4498077fd92ae5fcc13a4b749b96a4b7683e4db1c26b2c11d59 I0423 19:32:49.254885 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=cac8f2e4fc40c218d827bfd7d52339f45d8c9a2e8ac1aea95f287e04cf5e1911 I0423 19:32:49.254906 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0423 19:32:49.254923 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=3c27fec5d605e9857462ef0b374bea3d27a064363c8ab6ac4569bd2d9f9c1930 I0423 19:32:49.254931 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0423 19:32:49.254943 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=efcd8e1ed47960a7f1b746f2bef26d423d5cbb6adb68bca966500d8eb32a8117 I0423 19:32:49.254957 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=b3da34bcf041b2027c5e62fac8fac345d6b4ad6e5b759ab559141e5ee2ca22db I0423 19:32:49.254979 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=cf580199e23383b41f4c6403bd360a2fdc20cd4a8adacbfb2e1eb1081b5de262 I0423 19:32:49.254994 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=ff8e9902b5225ce906d4446c4b3c379ebf6e4742d5b7e9b21fe570ddde677297 I0423 19:32:49.255013 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=207fbbf38811baefa9335df011336c1d6e8c2eb63ec4518fc61a9152c87acc4d I0423 19:32:49.255022 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0423 19:32:49.255050 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=4aa9a7fb9433a1ee44e8e905e147838dac01ae47509736f23aa6293965f595ab I0423 19:32:49.255068 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0423 19:32:49.255076 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0423 19:32:49.255084 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.067845044s to process 36 records I0423 19:32:49.449652 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:32:49.449712 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-57ff957978-f8qz2 pod in namespace openshift-image-registry (previous: false). I0423 19:32:49.645646 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-57ff957978-f8qz2 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-57ff957978-f8qz2\" is waiting to start: ContainerCreating" I0423 19:32:49.645663 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-57ff957978-f8qz2\" is waiting to start: ContainerCreating" I0423 19:32:49.645716 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-57ff957978-zgs2j pod in namespace openshift-image-registry (previous: false). I0423 19:32:49.846542 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-57ff957978-zgs2j pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-57ff957978-zgs2j\" is waiting to start: ContainerCreating" I0423 19:32:49.846562 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-57ff957978-zgs2j\" is waiting to start: ContainerCreating" I0423 19:32:49.846597 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-dbb9d5bb5-nmrgk pod in namespace openshift-image-registry (previous: false). I0423 19:32:50.046713 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-dbb9d5bb5-nmrgk pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-dbb9d5bb5-nmrgk\" is waiting to start: ContainerCreating" I0423 19:32:50.046730 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-dbb9d5bb5-nmrgk\" is waiting to start: ContainerCreating" I0423 19:32:50.046741 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-dv4hr pod in namespace openshift-image-registry (previous: false). W0423 19:32:50.204377 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:32:50.250959 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:32:50.250981 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-kwddf pod in namespace openshift-image-registry (previous: false). I0423 19:32:50.449427 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:32:50.449443 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-l7zhz pod in namespace openshift-image-registry (previous: false). I0423 19:32:50.646818 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 19:32:50.646839 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5549b9c849-9xpw7 pod in namespace openshift-ingress (previous: false). I0423 19:32:50.846327 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5549b9c849-9xpw7 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5549b9c849-9xpw7\" is waiting to start: ContainerCreating" I0423 19:32:50.846347 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5549b9c849-9xpw7\" is waiting to start: ContainerCreating" I0423 19:32:50.846357 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5549b9c849-rjng6 pod in namespace openshift-ingress (previous: false). I0423 19:32:51.046891 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5549b9c849-rjng6 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5549b9c849-rjng6\" is waiting to start: ContainerCreating" I0423 19:32:51.046911 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5549b9c849-rjng6\" is waiting to start: ContainerCreating" I0423 19:32:51.046955 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-dbcccc67f-9st5g pod in namespace openshift-ingress (previous: false). W0423 19:32:51.205199 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 19:32:51.247296 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-dbcccc67f-9st5g pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-dbcccc67f-9st5g\" is waiting to start: ContainerCreating" I0423 19:32:51.247313 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-dbcccc67f-9st5g\" is waiting to start: ContainerCreating" I0423 19:32:51.247324 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-2hfdj pod in namespace openshift-ingress-canary (previous: false). I0423 19:32:51.452656 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-cpc9k pod in namespace openshift-ingress-canary (previous: false). I0423 19:32:51.657873 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dr2mb pod in namespace openshift-ingress-canary (previous: false). I0423 19:32:51.848089 1 tasks_processing.go:74] worker 7 stopped. I0423 19:32:51.848187 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=13cc69e066993b4f5a03f85e3d3fdf0da95bcafc1ee13ae1b4b2f5fa9278582e I0423 19:32:51.848237 1 recorder.go:75] Recording events/openshift-dns with fingerprint=3a9f0b6cfe6405d0b4dfe71473b45550ee9779ecdaedab81e984de2a3f3481e5 I0423 19:32:51.848319 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=4bab24ba878838f9278cf1ca72ccdfbcdc4481f86f1085143f9450b7d8ec8f1b I0423 19:32:51.848348 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=63058122684fcf45844f3cb8a4d7b7fc092c549f0a10b9b59dbcc611715e4df6 I0423 19:32:51.848402 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=aee028e1633bb1427b5054cd50415071be6fed47fc15b69fd1fd3f164b80e49b I0423 19:32:51.848438 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=609d96af8d0964596dbd102526b8beb18f45820e252c4bd1f9e38337df3848b7 I0423 19:32:51.849758 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-ln9x5 with fingerprint=3772c680e03ac24ab06061fbfc35536c0024fcc5a31965c0f3d6915197b91b08 I0423 19:32:51.849850 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-t9t42 with fingerprint=7ac8a831814c95d895666e305438508f8b87a167e83c8053fa1c4a02b5ea33ac I0423 19:32:51.849921 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-wghf4 with fingerprint=d53bf3b0b1d6073d4fe0b1909043e366ec5d80394bddb50d119334e8a5ebcf96 I0423 19:32:51.850021 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-57ff957978-f8qz2 with fingerprint=26363a19fa636748fe64c661cfa4bf9c1c6ac3e508cb0487c844dc04e943a564 I0423 19:32:51.850110 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-57ff957978-zgs2j with fingerprint=b00d99e7a9a03ba5b05baef11d8a8e0d011efb81d777356fe9b165feced1197d I0423 19:32:51.850209 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-dbb9d5bb5-nmrgk with fingerprint=57e696184da33555ce8be2fd2c7fa3636c82b50e3ddd6feb126eb6d47b15149d I0423 19:32:51.850217 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-2hfdj/serve-healthcheck-canary_current.log with fingerprint=7c72ff8cf9b97b3ec75683920be4ffd6f4f5197668881f69dad8609600cea479 I0423 19:32:51.850221 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-cpc9k/serve-healthcheck-canary_current.log with fingerprint=4b81bd767891dcfe915d67d93a23640270c25751c08a188ac63ef2496305ef72 I0423 19:32:51.850225 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-dr2mb/serve-healthcheck-canary_current.log with fingerprint=208bddbbaff5fec6b081cc97944e7d86c3370d957eae55f48dab11fccc54b63b I0423 19:32:51.850232 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.663060283s to process 15 records W0423 19:32:52.204161 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0423 19:32:52.204184 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0423 19:32:52.204201 1 tasks_processing.go:74] worker 48 stopped. E0423 19:32:52.204212 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0423 19:32:52.204224 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0423 19:32:52.204238 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0423 19:32:52.204250 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.019181359s to process 1 records I0423 19:32:59.640984 1 tasks_processing.go:74] worker 28 stopped. I0423 19:32:59.641029 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0423 19:32:59.641043 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.453032581s to process 1 records I0423 19:32:59.964161 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:33:00.394014 1 tasks_processing.go:74] worker 10 stopped. I0423 19:33:00.394322 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=8e51bdad94ccef4d8b8d1dcd31ca57822f1858ab42bfebeef1b93869b7cdc5cb I0423 19:33:00.394342 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.206047128s to process 1 records E0423 19:33:00.394397 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.209s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0423 19:33:00.395504 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0423 19:33:00.395515 1 periodic.go:216] Running workloads gatherer I0423 19:33:00.395529 1 tasks_processing.go:45] number of workers: 2 I0423 19:33:00.395536 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 19:33:00.395540 1 tasks_processing.go:71] worker 1 working on workload_info task. I0423 19:33:00.395548 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 19:33:00.395576 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0423 19:33:00.420551 1 tasks_processing.go:74] worker 0 stopped. I0423 19:33:00.420567 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 24.953856ms to process 0 records I0423 19:33:00.421397 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0423 19:33:00.431537 1 gather_workloads_info.go:387] No image sha256:a043239802b3eb8b323d285193d2527fad0ecec98ca91d188a3472a2fac8ae04 (11ms) I0423 19:33:00.441596 1 gather_workloads_info.go:387] No image sha256:730d1b6988025bef0daa3a9a5d8467ec4a26b0382cc52f91c3375b4590d3518a (10ms) I0423 19:33:00.457446 1 gather_workloads_info.go:387] No image sha256:90a8ffd9643ebb16a6a8c04bb38cf9ed58903e9d3bf836c68f399193db5edaf6 (16ms) I0423 19:33:00.466897 1 gather_workloads_info.go:387] No image sha256:e8b96d9318b3b8c9ed0afe4e6381f635c6b0c2f20772044ac68001ed49af2c87 (9ms) I0423 19:33:00.475345 1 gather_workloads_info.go:387] No image sha256:3bba1358d4a0ae878ff491c0c2cbfffe60649e110b40342b878fe8fa332f8858 (8ms) I0423 19:33:00.484598 1 gather_workloads_info.go:387] No image sha256:875c77e5d144f03fb91d8cee0259f6966683ca88d1bf818dbf4652c16b70312c (9ms) I0423 19:33:00.493033 1 gather_workloads_info.go:387] No image sha256:d6fbe0075cbb12bfd287c973704fadf97154c7f73e370733d976a40835e9155a (8ms) I0423 19:33:00.502344 1 gather_workloads_info.go:387] No image sha256:25c148fd380b1a9db3f6039d2e0eabc489a954921452391390ba9192b2325678 (9ms) I0423 19:33:00.511256 1 gather_workloads_info.go:387] No image sha256:08c5a78c8a5af04c549e2273aaf4bb452a75bf038d68aa9d01bb2aff66c30e90 (9ms) I0423 19:33:00.519746 1 gather_workloads_info.go:387] No image sha256:4733236617781e3469ffffb15e4daaa1f14ea8e1c52b426a3787a4f1f2945424 (8ms) I0423 19:33:00.528738 1 gather_workloads_info.go:387] No image sha256:ab60623bb32f7e75fca71ef65137731cae347a21c7a4091dfd583fa00732721c (9ms) I0423 19:33:00.632786 1 gather_workloads_info.go:387] No image sha256:55b1db6038c5beaed54c626e3343b7a8589cc0be8dc41d1a66b4deab766ff520 (104ms) I0423 19:33:00.731022 1 gather_workloads_info.go:387] No image sha256:5ac9c549d65fc1d8bc900773bebee43e9192bcec1bb5fa46afb4597230c16ac7 (98ms) I0423 19:33:00.830431 1 gather_workloads_info.go:387] No image sha256:5c6d21c3f97366bc7ab57031cc027b67405a684bd804ce364ed5998b0685eaca (99ms) I0423 19:33:00.931328 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (101ms) I0423 19:33:01.030872 1 gather_workloads_info.go:387] No image sha256:ce138e8cf5b96557b1864ef6f27c2608bdca59be2611804366cef7169c36291e (100ms) I0423 19:33:01.130389 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (99ms) I0423 19:33:01.231296 1 gather_workloads_info.go:387] No image sha256:0a4dfb8d4c1b3849319d45b4c54dff26a7238a2c08fcaa121f93073e95ab12e8 (101ms) I0423 19:33:01.331662 1 gather_workloads_info.go:387] No image sha256:e1ba458cf6f0b3606c90880da72db8ab99cd11040bae84baebe3ff2e0d1ea075 (100ms) I0423 19:33:01.430045 1 gather_workloads_info.go:387] No image sha256:5808401268394502d335281ea1a294b07210461b986b58f91d7d1f29c0029c6d (98ms) I0423 19:33:01.530998 1 gather_workloads_info.go:387] No image sha256:89277d8d4560d71db88c2dcc67c992a24544ca21810920b609c1d49d53b4a287 (101ms) I0423 19:33:01.630660 1 gather_workloads_info.go:387] No image sha256:e3732e356ae2324565c74cae57d5d016917314fa293ddaa3a68ee9ae030c6f07 (100ms) I0423 19:33:01.731350 1 gather_workloads_info.go:387] No image sha256:5aaea0419169e55832cc27acfe0fe3b9513a343d6bdf71d3da1575ed322245d0 (101ms) I0423 19:33:01.826426 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:33:01.830252 1 gather_workloads_info.go:387] No image sha256:30e597ec5d6bb96ff70a4f8688c748b659cd4fd5d73d222e8701821d236795c5 (99ms) I0423 19:33:01.930952 1 gather_workloads_info.go:387] No image sha256:15677f0b70e6aa2dfaf088e45fc1a425c22bf6fda326b8116f87e88b6694dfab (101ms) I0423 19:33:02.025439 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:33:02.030312 1 gather_workloads_info.go:387] No image sha256:48e883a3932aea9457f55cd4628d21397d429d4828f42f95d1c903d9d5395bde (99ms) I0423 19:33:02.131177 1 gather_workloads_info.go:387] No image sha256:934d8e8c50f3c609b8eea80d1051111fe3d066fe8c65c79572072ae55fcb0a86 (101ms) I0423 19:33:02.230504 1 gather_workloads_info.go:387] No image sha256:6814f5fced0ef219d06374011c68a11a3da788a764a00a69ade435466d9ee240 (99ms) I0423 19:33:02.330488 1 gather_workloads_info.go:387] No image sha256:673ebc8cc22c56c8f410e011b2fa950d28cf7b6420e17fdb6580d6cb10523384 (100ms) I0423 19:33:02.431333 1 gather_workloads_info.go:387] No image sha256:d64bea34bf3e1bb0b3a701c3ff14e66665afc1b050f28124ad7e6888eaec3a81 (101ms) I0423 19:33:02.531300 1 gather_workloads_info.go:387] No image sha256:084aa9b0f8a6d478549dc384d4e66da13ee9b25cc98531da861cc19dee2a9e8f (100ms) I0423 19:33:02.630914 1 gather_workloads_info.go:387] No image sha256:5b6212b8f539f08e78417d8a4b7485ca0b4e7927cacd7b752742a28841bc8ccd (100ms) I0423 19:33:02.732187 1 gather_workloads_info.go:387] No image sha256:000105ef5150e7079b90a613fb9e6193e2a6ef9b1908d2dce44f2395d4fd070f (101ms) I0423 19:33:02.830789 1 gather_workloads_info.go:387] No image sha256:637d41f067a5239096fc22b135181cda5113da833f1370e7a73965e83792e93a (99ms) I0423 19:33:02.830820 1 tasks_processing.go:74] worker 1 stopped. E0423 19:33:02.830829 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0423 19:33:02.831082 1 recorder.go:75] Recording config/workload_info with fingerprint=8ef2aca6415ac4bca79fef2e1b0249842090d047673cd554c0501df581a390f0 I0423 19:33:02.831098 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.435272983s to process 1 records E0423 19:33:02.831131 1 periodic.go:254] "Unhandled Error" err="workloads failed after 2.435s with: function \"workload_info\" failed with an error" I0423 19:33:02.832256 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0423 19:33:02.832270 1 periodic.go:216] Running conditional gatherer I0423 19:33:02.837692 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules I0423 19:33:02.844269 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.16:41112->172.30.0.10:53: read: connection refused E0423 19:33:02.844590 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0423 19:33:02.844644 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0423 19:33:02.850091 1 conditional_gatherer.go:392] cluster version is '4.21.9' E0423 19:33:02.850105 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850110 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850113 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850116 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850119 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850123 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850126 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850128 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0423 19:33:02.850131 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0423 19:33:02.850164 1 tasks_processing.go:45] number of workers: 3 I0423 19:33:02.850175 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 19:33:02.850179 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0423 19:33:02.850186 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 19:33:02.850194 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 19:33:02.850215 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0423 19:33:02.850219 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0423 19:33:02.850236 1 tasks_processing.go:74] worker 1 stopped. I0423 19:33:02.850326 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0423 19:33:02.850344 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.536µs to process 1 records I0423 19:33:02.850384 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0423 19:33:02.850394 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.479µs to process 1 records I0423 19:33:02.850403 1 tasks_processing.go:74] worker 0 stopped. I0423 19:33:02.850411 1 tasks_processing.go:74] worker 2 stopped. I0423 19:33:02.850421 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 227.629µs to process 0 records I0423 19:33:02.850439 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.16:41112->172.30.0.10:53: read: connection refused I0423 19:33:02.850458 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0423 19:33:02.871831 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=72f901889373da229e4c26a5efea6b56212d3bb800f9e85f00656b3bd0901ad4 I0423 19:33:02.871956 1 diskrecorder.go:70] Writing 110 records to /var/lib/insights-operator/insights-2026-04-23-193302.tar.gz I0423 19:33:02.878317 1 diskrecorder.go:51] Wrote 110 records to disk in 6ms I0423 19:33:02.878346 1 periodic.go:285] Gathering cluster info every 2h0m0s I0423 19:33:02.878361 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0423 19:33:13.668708 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 19:34:06.650494 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="dd51c31e195f66ed995426aa5fdc4d57099ade1b835313be392d589da0b9b55f") W0423 19:34:06.650524 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.key was created I0423 19:34:06.650576 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0423 19:34:06.650606 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="cb7d041ea28ef9379c9f9d5fdc0a23ffb8932b221b7983087f31a1c0abfaf033") I0423 19:34:06.650618 1 base_controller.go:181] Shutting down LoggingSyncer ... I0423 19:34:06.650633 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0423 19:34:06.650645 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0423 19:34:06.650651 1 base_controller.go:113] All LoggingSyncer workers have been terminated I0423 19:34:06.650685 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped"