W0327 15:10:01.953076 1 cmd.go:257] Using insecure, self-signed certificates I0327 15:10:02.766745 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0327 15:10:02.767049 1 observer_polling.go:159] Starting file observer I0327 15:10:03.753128 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0327 15:10:03.753426 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0327 15:10:03.754382 1 secure_serving.go:57] Forcing use of http/1.1 only W0327 15:10:03.754410 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0327 15:10:03.754417 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0327 15:10:03.754423 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0327 15:10:03.754427 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0327 15:10:03.754430 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0327 15:10:03.754434 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0327 15:10:03.754386 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0327 15:10:03.758151 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0327 15:10:03.758230 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"ddf6bb33-2554-423c-82c8-dd5994810db2", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0327 15:10:03.759072 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0327 15:10:03.759091 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0327 15:10:03.759091 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0327 15:10:03.759093 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0327 15:10:03.759127 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0327 15:10:03.759131 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0327 15:10:03.759748 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1042587149/tls.crt::/tmp/serving-cert-1042587149/tls.key" I0327 15:10:03.760332 1 secure_serving.go:213] Serving securely on [::]:8443 I0327 15:10:03.760514 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0327 15:10:03.762660 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0327 15:10:03.762682 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0327 15:10:03.762778 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0327 15:10:03.767228 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 15:10:03.767244 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0327 15:10:03.771151 1 secretconfigobserver.go:119] support secret does not exist I0327 15:10:03.774690 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0327 15:10:03.778084 1 secretconfigobserver.go:119] support secret does not exist I0327 15:10:03.781084 1 recorder.go:176] Pruning old reports every 4h4m54s, max age is 288h0m0s I0327 15:10:03.785021 1 periodic.go:216] Running clusterconfig gatherer I0327 15:10:03.785029 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0327 15:10:03.785043 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0327 15:10:03.785059 1 tasks_processing.go:45] number of workers: 64 I0327 15:10:03.785061 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0327 15:10:03.785069 1 insightsreport.go:296] Starting report retriever I0327 15:10:03.785075 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0327 15:10:03.785079 1 tasks_processing.go:69] worker 4 listening for tasks. I0327 15:10:03.785086 1 tasks_processing.go:69] worker 2 listening for tasks. I0327 15:10:03.785087 1 tasks_processing.go:69] worker 3 listening for tasks. I0327 15:10:03.785086 1 tasks_processing.go:69] worker 1 listening for tasks. I0327 15:10:03.785096 1 tasks_processing.go:69] worker 37 listening for tasks. I0327 15:10:03.785098 1 tasks_processing.go:69] worker 26 listening for tasks. I0327 15:10:03.785100 1 tasks_processing.go:69] worker 31 listening for tasks. I0327 15:10:03.785102 1 tasks_processing.go:69] worker 32 listening for tasks. I0327 15:10:03.785101 1 tasks_processing.go:69] worker 0 listening for tasks. I0327 15:10:03.785109 1 tasks_processing.go:69] worker 38 listening for tasks. I0327 15:10:03.785111 1 tasks_processing.go:69] worker 34 listening for tasks. I0327 15:10:03.785111 1 tasks_processing.go:69] worker 27 listening for tasks. I0327 15:10:03.785117 1 tasks_processing.go:69] worker 39 listening for tasks. I0327 15:10:03.785111 1 tasks_processing.go:71] worker 32 working on image_pruners task. I0327 15:10:03.785121 1 tasks_processing.go:69] worker 35 listening for tasks. I0327 15:10:03.785122 1 tasks_processing.go:69] worker 5 listening for tasks. I0327 15:10:03.785125 1 tasks_processing.go:71] worker 39 working on image_registries task. I0327 15:10:03.785128 1 tasks_processing.go:69] worker 30 listening for tasks. I0327 15:10:03.785131 1 tasks_processing.go:69] worker 10 listening for tasks. I0327 15:10:03.785130 1 tasks_processing.go:69] worker 41 listening for tasks. I0327 15:10:03.785104 1 tasks_processing.go:71] worker 26 working on authentication task. I0327 15:10:03.785137 1 tasks_processing.go:69] worker 11 listening for tasks. I0327 15:10:03.785139 1 tasks_processing.go:71] worker 41 working on mutating_webhook_configurations task. I0327 15:10:03.785141 1 tasks_processing.go:69] worker 43 listening for tasks. I0327 15:10:03.785142 1 tasks_processing.go:69] worker 36 listening for tasks. I0327 15:10:03.785148 1 tasks_processing.go:69] worker 12 listening for tasks. I0327 15:10:03.785152 1 tasks_processing.go:69] worker 44 listening for tasks. I0327 15:10:03.785155 1 tasks_processing.go:69] worker 13 listening for tasks. I0327 15:10:03.785160 1 tasks_processing.go:71] worker 13 working on crds task. I0327 15:10:03.785157 1 tasks_processing.go:69] worker 21 listening for tasks. I0327 15:10:03.785166 1 tasks_processing.go:69] worker 46 listening for tasks. I0327 15:10:03.785168 1 tasks_processing.go:71] worker 12 working on clusterroles task. I0327 15:10:03.785175 1 tasks_processing.go:69] worker 16 listening for tasks. I0327 15:10:03.785176 1 tasks_processing.go:69] worker 47 listening for tasks. I0327 15:10:03.785136 1 tasks_processing.go:69] worker 42 listening for tasks. I0327 15:10:03.785143 1 tasks_processing.go:71] worker 11 working on metrics task. I0327 15:10:03.785176 1 tasks_processing.go:71] worker 38 working on openstack_version task. I0327 15:10:03.785189 1 tasks_processing.go:69] worker 18 listening for tasks. I0327 15:10:03.785125 1 tasks_processing.go:69] worker 40 listening for tasks. I0327 15:10:03.785135 1 tasks_processing.go:71] worker 30 working on proxies task. W0327 15:10:03.785213 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0327 15:10:03.785106 1 tasks_processing.go:69] worker 33 listening for tasks. I0327 15:10:03.785230 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 34.794µs to process 0 records I0327 15:10:03.785240 1 tasks_processing.go:69] worker 48 listening for tasks. I0327 15:10:03.785103 1 tasks_processing.go:71] worker 37 working on machine_config_pools task. I0327 15:10:03.785153 1 tasks_processing.go:71] worker 36 working on dvo_metrics task. I0327 15:10:03.785114 1 tasks_processing.go:69] worker 15 listening for tasks. I0327 15:10:03.785351 1 tasks_processing.go:69] worker 19 listening for tasks. I0327 15:10:03.785357 1 tasks_processing.go:69] worker 20 listening for tasks. I0327 15:10:03.785358 1 tasks_processing.go:69] worker 7 listening for tasks. I0327 15:10:03.785362 1 tasks_processing.go:71] worker 4 working on certificate_signing_requests task. I0327 15:10:03.785371 1 tasks_processing.go:69] worker 61 listening for tasks. I0327 15:10:03.785377 1 tasks_processing.go:69] worker 62 listening for tasks. I0327 15:10:03.785384 1 tasks_processing.go:69] worker 63 listening for tasks. I0327 15:10:03.785123 1 tasks_processing.go:69] worker 29 listening for tasks. I0327 15:10:03.785392 1 tasks_processing.go:69] worker 6 listening for tasks. I0327 15:10:03.785184 1 tasks_processing.go:69] worker 17 listening for tasks. I0327 15:10:03.785392 1 tasks_processing.go:69] worker 54 listening for tasks. I0327 15:10:03.785400 1 tasks_processing.go:71] worker 3 working on sap_datahubs task. I0327 15:10:03.785409 1 tasks_processing.go:69] worker 50 listening for tasks. I0327 15:10:03.785407 1 tasks_processing.go:69] worker 59 listening for tasks. I0327 15:10:03.785417 1 tasks_processing.go:69] worker 55 listening for tasks. I0327 15:10:03.785419 1 tasks_processing.go:69] worker 49 listening for tasks. I0327 15:10:03.785408 1 tasks_processing.go:69] worker 56 listening for tasks. I0327 15:10:03.785425 1 tasks_processing.go:69] worker 57 listening for tasks. I0327 15:10:03.785425 1 tasks_processing.go:69] worker 52 listening for tasks. I0327 15:10:03.785418 1 tasks_processing.go:69] worker 51 listening for tasks. I0327 15:10:03.785434 1 tasks_processing.go:69] worker 53 listening for tasks. I0327 15:10:03.785125 1 tasks_processing.go:71] worker 35 working on sap_config task. I0327 15:10:03.785441 1 tasks_processing.go:69] worker 22 listening for tasks. I0327 15:10:03.785442 1 tasks_processing.go:69] worker 58 listening for tasks. I0327 15:10:03.785447 1 tasks_processing.go:69] worker 23 listening for tasks. I0327 15:10:03.785450 1 tasks_processing.go:71] worker 22 working on operators task. I0327 15:10:03.785741 1 tasks_processing.go:71] worker 10 working on nodenetworkconfigurationpolicies task. I0327 15:10:03.785806 1 tasks_processing.go:71] worker 23 working on operators_pods_and_events task. I0327 15:10:03.785894 1 tasks_processing.go:71] worker 59 working on storage_cluster task. I0327 15:10:03.785935 1 tasks_processing.go:71] worker 6 working on sap_pods task. I0327 15:10:03.785990 1 tasks_processing.go:71] worker 54 working on machine_configs task. I0327 15:10:03.786062 1 tasks_processing.go:71] worker 51 working on lokistack task. I0327 15:10:03.786107 1 tasks_processing.go:71] worker 53 working on ceph_cluster task. I0327 15:10:03.786175 1 tasks_processing.go:71] worker 62 working on olm_operators task. I0327 15:10:03.785115 1 tasks_processing.go:71] worker 0 working on machines task. I0327 15:10:03.786084 1 tasks_processing.go:71] worker 52 working on silenced_alerts task. I0327 15:10:03.785807 1 tasks_processing.go:71] worker 16 working on infrastructures task. W0327 15:10:03.786610 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0327 15:10:03.786623 1 tasks_processing.go:71] worker 52 working on openshift_logging task. I0327 15:10:03.785769 1 tasks_processing.go:71] worker 40 working on pod_network_connectivity_checks task. I0327 15:10:03.785942 1 tasks_processing.go:71] worker 7 working on container_images task. I0327 15:10:03.785919 1 tasks_processing.go:71] worker 18 working on openshift_machine_api_events task. I0327 15:10:03.786864 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 51.202µs to process 0 records I0327 15:10:03.786146 1 tasks_processing.go:71] worker 63 working on openstack_dataplanedeployments task. I0327 15:10:03.785160 1 tasks_processing.go:69] worker 45 listening for tasks. I0327 15:10:03.787014 1 tasks_processing.go:71] worker 45 working on cluster_apiserver task. I0327 15:10:03.785160 1 tasks_processing.go:71] worker 44 working on networks task. I0327 15:10:03.785147 1 tasks_processing.go:71] worker 43 working on support_secret task. I0327 15:10:03.785450 1 tasks_processing.go:71] worker 2 working on machine_sets task. I0327 15:10:03.787140 1 tasks_processing.go:69] worker 8 listening for tasks. I0327 15:10:03.787148 1 tasks_processing.go:69] worker 14 listening for tasks. I0327 15:10:03.787153 1 tasks_processing.go:71] worker 14 working on node_features task. I0327 15:10:03.787153 1 tasks_processing.go:71] worker 8 working on machine_healthchecks task. I0327 15:10:03.785434 1 tasks_processing.go:69] worker 24 listening for tasks. I0327 15:10:03.785429 1 tasks_processing.go:71] worker 1 working on cost_management_metrics_configs task. I0327 15:10:03.785447 1 tasks_processing.go:71] worker 34 working on nodenetworkstates task. I0327 15:10:03.785444 1 tasks_processing.go:71] worker 31 working on openstack_dataplanenodesets task. I0327 15:10:03.787410 1 tasks_processing.go:71] worker 24 working on qemu_kubevirt_launcher_logs task. I0327 15:10:03.785366 1 tasks_processing.go:69] worker 60 listening for tasks. I0327 15:10:03.785117 1 tasks_processing.go:69] worker 28 listening for tasks. I0327 15:10:03.787488 1 tasks_processing.go:71] worker 28 working on storage_classes task. I0327 15:10:03.785130 1 tasks_processing.go:71] worker 5 working on jaegers task. I0327 15:10:03.785453 1 tasks_processing.go:69] worker 25 listening for tasks. I0327 15:10:03.785772 1 tasks_processing.go:71] worker 58 working on oauths task. I0327 15:10:03.785795 1 tasks_processing.go:71] worker 27 working on pdbs task. I0327 15:10:03.785803 1 tasks_processing.go:71] worker 19 working on config_maps task. I0327 15:10:03.785805 1 tasks_processing.go:71] worker 15 working on node_logs task. I0327 15:10:03.785818 1 tasks_processing.go:71] worker 11 working on container_runtime_configs task. I0327 15:10:03.788204 1 tasks_processing.go:71] worker 25 working on install_plans task. I0327 15:10:03.785821 1 tasks_processing.go:71] worker 46 working on tsdb_status task. W0327 15:10:03.788234 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0327 15:10:03.788245 1 tasks_processing.go:71] worker 46 working on service_accounts task. I0327 15:10:03.785824 1 tasks_processing.go:71] worker 33 working on aggregated_monitoring_cr_names task. I0327 15:10:03.788293 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 25.313µs to process 0 records I0327 15:10:03.785849 1 tasks_processing.go:71] worker 47 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0327 15:10:03.785856 1 tasks_processing.go:71] worker 20 working on monitoring_persistent_volumes task. I0327 15:10:03.785855 1 tasks_processing.go:71] worker 42 working on validating_webhook_configurations task. I0327 15:10:03.785923 1 tasks_processing.go:71] worker 21 working on feature_gates task. I0327 15:10:03.785962 1 tasks_processing.go:71] worker 48 working on overlapping_namespace_uids task. I0327 15:10:03.785970 1 tasks_processing.go:71] worker 49 working on image task. I0327 15:10:03.785975 1 tasks_processing.go:71] worker 55 working on nodes task. I0327 15:10:03.785994 1 tasks_processing.go:71] worker 17 working on version task. I0327 15:10:03.786020 1 tasks_processing.go:71] worker 50 working on schedulers task. I0327 15:10:03.786022 1 tasks_processing.go:71] worker 57 working on ingress_certificates task. I0327 15:10:03.786091 1 tasks_processing.go:71] worker 56 working on active_alerts task. I0327 15:10:03.786971 1 tasks_processing.go:71] worker 29 working on openstack_controlplanes task. W0327 15:10:03.788878 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0327 15:10:03.788889 1 tasks_processing.go:74] worker 56 stopped. I0327 15:10:03.788900 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 25.889µs to process 0 records I0327 15:10:03.787477 1 tasks_processing.go:71] worker 60 working on ingress task. I0327 15:10:03.785768 1 tasks_processing.go:71] worker 61 working on machine_autoscalers task. I0327 15:10:03.785768 1 tasks_processing.go:69] worker 9 listening for tasks. I0327 15:10:03.789000 1 tasks_processing.go:74] worker 9 stopped. I0327 15:10:03.790247 1 tasks_processing.go:74] worker 37 stopped. I0327 15:10:03.790264 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 4.97347ms to process 0 records I0327 15:10:03.790351 1 tasks_processing.go:74] worker 38 stopped. I0327 15:10:03.790363 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 5.156157ms to process 0 records I0327 15:10:03.790441 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0327 15:10:03.790457 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0327 15:10:03.790462 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0327 15:10:03.790465 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0327 15:10:03.790478 1 controller.go:531] The operator is still being initialized I0327 15:10:03.790485 1 controller.go:554] The operator is healthy I0327 15:10:03.791260 1 tasks_processing.go:74] worker 26 stopped. I0327 15:10:03.791455 1 recorder.go:75] Recording config/authentication with fingerprint=e56694408c757e1eab5da821b7d84af11a0d8371f876e1fc2bb97a1b8cc6d2bf I0327 15:10:03.791469 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 6.11231ms to process 1 records I0327 15:10:03.791478 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 5.175841ms to process 0 records I0327 15:10:03.791482 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 5.889985ms to process 0 records I0327 15:10:03.791487 1 tasks_processing.go:74] worker 3 stopped. I0327 15:10:03.791493 1 tasks_processing.go:74] worker 51 stopped. I0327 15:10:03.791926 1 tasks_processing.go:74] worker 6 stopped. I0327 15:10:03.791939 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 5.927674ms to process 0 records I0327 15:10:03.792049 1 tasks_processing.go:74] worker 30 stopped. I0327 15:10:03.792145 1 recorder.go:75] Recording config/proxy with fingerprint=64b63bb82b0b965706ea13ae646c3b549eee7ff0f0661b117c8bc769c3e1dd94 I0327 15:10:03.792162 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 6.838343ms to process 1 records E0327 15:10:03.792178 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0327 15:10:03.792190 1 gather.go:177] gatherer "clusterconfig" function "machines" took 5.671999ms to process 0 records I0327 15:10:03.792199 1 tasks_processing.go:74] worker 0 stopped. I0327 15:10:03.792315 1 tasks_processing.go:74] worker 32 stopped. I0327 15:10:03.792511 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=4bc6c2f3e2494e7d1c0394c521354b75e4c17197b48f1846f605867c1d83aebe I0327 15:10:03.792523 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 7.183166ms to process 1 records I0327 15:10:03.796692 1 tasks_processing.go:74] worker 40 stopped. E0327 15:10:03.796703 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0327 15:10:03.796710 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 9.979612ms to process 0 records I0327 15:10:03.796716 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 9.334476ms to process 0 records I0327 15:10:03.796721 1 tasks_processing.go:74] worker 34 stopped. I0327 15:10:03.796747 1 tasks_processing.go:74] worker 63 stopped. I0327 15:10:03.796756 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 9.798944ms to process 0 records I0327 15:10:03.796772 1 tasks_processing.go:74] worker 52 stopped. I0327 15:10:03.796784 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 10.139884ms to process 0 records I0327 15:10:03.796796 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 8.633231ms to process 0 records I0327 15:10:03.796804 1 tasks_processing.go:74] worker 11 stopped. I0327 15:10:03.797090 1 tasks_processing.go:74] worker 31 stopped. I0327 15:10:03.797100 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 9.674564ms to process 0 records I0327 15:10:03.809801 1 tasks_processing.go:74] worker 35 stopped. I0327 15:10:03.809813 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 24.353182ms to process 0 records I0327 15:10:03.809829 1 tasks_processing.go:74] worker 10 stopped. I0327 15:10:03.809839 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 24.006986ms to process 0 records E0327 15:10:03.809844 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0327 15:10:03.809849 1 tasks_processing.go:74] worker 14 stopped. I0327 15:10:03.809854 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 22.691625ms to process 0 records I0327 15:10:03.809954 1 tasks_processing.go:74] worker 5 stopped. I0327 15:10:03.809963 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 22.450851ms to process 0 records I0327 15:10:03.809982 1 tasks_processing.go:74] worker 18 stopped. I0327 15:10:03.809990 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 23.163175ms to process 0 records I0327 15:10:03.810557 1 tasks_processing.go:74] worker 20 stopped. I0327 15:10:03.810574 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 22.09397ms to process 0 records E0327 15:10:03.810600 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0327 15:10:03.810608 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 23.492869ms to process 0 records I0327 15:10:03.810614 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 23.450742ms to process 0 records I0327 15:10:03.810620 1 tasks_processing.go:74] worker 2 stopped. I0327 15:10:03.810620 1 tasks_processing.go:74] worker 43 stopped. I0327 15:10:03.810641 1 tasks_processing.go:74] worker 16 stopped. I0327 15:10:03.811262 1 recorder.go:75] Recording config/infrastructure with fingerprint=b6d6b02e3e9df63334a0313d46c5d87eb9c2cee1090bc4d95c075015dfdcf65b I0327 15:10:03.811282 1 gather_logs.go:145] no pods in namespace were found I0327 15:10:03.811765 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 24.057262ms to process 1 records I0327 15:10:03.811956 1 tasks_processing.go:74] worker 49 stopped. I0327 15:10:03.811968 1 recorder.go:75] Recording config/image with fingerprint=e3714158c864fb93aa86a9358ba667c3cff0f92abccb31595ec7b76a7634159a I0327 15:10:03.811991 1 gather.go:177] gatherer "clusterconfig" function "image" took 22.031725ms to process 1 records I0327 15:10:03.812000 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 25.012695ms to process 0 records I0327 15:10:03.812040 1 tasks_processing.go:74] worker 59 stopped. I0327 15:10:03.812213 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=90f7d4f2cfbfce50ea81f8f4759b2b428fd60a14eec444edee21533cda489252 I0327 15:10:03.812267 1 tasks_processing.go:74] worker 28 stopped. I0327 15:10:03.812284 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=ea642d4044da64f414f00c4265f4e6e4ead900b3cfca358a06a8560964251ce5 I0327 15:10:03.812303 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 23.581439ms to process 2 records I0327 15:10:03.812310 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 22.197015ms to process 0 records I0327 15:10:03.812319 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 22.32777ms to process 0 records I0327 15:10:03.812327 1 tasks_processing.go:74] worker 61 stopped. I0327 15:10:03.812350 1 tasks_processing.go:74] worker 29 stopped. I0327 15:10:03.812483 1 tasks_processing.go:74] worker 44 stopped. I0327 15:10:03.812639 1 recorder.go:75] Recording config/network with fingerprint=c45972dbd95e09ece080d02d79347e566ac89e144f687643a8d6c5cc6173e6b8 I0327 15:10:03.812656 1 gather.go:177] gatherer "clusterconfig" function "networks" took 24.169808ms to process 1 records I0327 15:10:03.812664 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 23.226243ms to process 0 records I0327 15:10:03.812680 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 24.050235ms to process 0 records I0327 15:10:03.812687 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 25.431371ms to process 0 records I0327 15:10:03.812772 1 tasks_processing.go:74] worker 24 stopped. I0327 15:10:03.812787 1 tasks_processing.go:74] worker 15 stopped. I0327 15:10:03.812798 1 tasks_processing.go:74] worker 53 stopped. I0327 15:10:03.812872 1 tasks_processing.go:74] worker 45 stopped. I0327 15:10:03.812991 1 recorder.go:75] Recording config/apiserver with fingerprint=bfd61244cec117de56af4e3099d27d25cd696ac39232c331a52b9b7da2a5c8ff I0327 15:10:03.813012 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 25.169992ms to process 1 records I0327 15:10:03.813031 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 25.009744ms to process 0 records I0327 15:10:03.813494 1 recorder.go:75] Recording config/oauth with fingerprint=c56ded58326bc77e2dbc0b89d26e7370185e1976a0b5124bec7c472e5b5be718 I0327 15:10:03.813554 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 25.111372ms to process 1 records I0327 15:10:03.813606 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 26.587125ms to process 0 records I0327 15:10:03.813498 1 tasks_processing.go:74] worker 1 stopped. I0327 15:10:03.813511 1 tasks_processing.go:74] worker 58 stopped. I0327 15:10:03.813620 1 tasks_processing.go:74] worker 62 stopped. I0327 15:10:03.813864 1 tasks_processing.go:74] worker 41 stopped. I0327 15:10:03.814432 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=e32b2ad04bd933e5c825801844c7694083bc757be235e4f3a0684780c063e805 I0327 15:10:03.814529 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=a8689d9f3b77d5b3a0f664621765e3f676ee3a6cbce0a830b60f16a351badd1e I0327 15:10:03.814670 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=4ef183cb7ced68cfa2f38eb6285e04cfef92ac7f24d9458311579485bfda240e I0327 15:10:03.815139 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 28.713565ms to process 3 records E0327 15:10:03.815152 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0327 15:10:03.815161 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 26.732016ms to process 0 records I0327 15:10:03.815181 1 tasks_processing.go:74] worker 8 stopped. I0327 15:10:03.815447 1 tasks_processing.go:74] worker 21 stopped. I0327 15:10:03.815687 1 recorder.go:75] Recording config/featuregate with fingerprint=a358674c6e9dd8a7e21f421d361375a26d837252b2c3c49b1e027638f184937d I0327 15:10:03.815705 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 26.838654ms to process 1 records I0327 15:10:03.815953 1 tasks_processing.go:74] worker 27 stopped. I0327 15:10:03.816064 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=19957c8223973b2069a10f8ab3330f7986bcde63d7805c2995a82a02d8225367 I0327 15:10:03.816094 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=4a91a1a566d00d7cab153b855a1901fc301a9962924a70512f9c8b7e7e7c62dc I0327 15:10:03.816118 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=a881f74e548933607c9717367433475dd3103a5c2aa67e223a24fab4ffb4107b I0327 15:10:03.816131 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 28.288384ms to process 3 records I0327 15:10:03.816495 1 tasks_processing.go:74] worker 60 stopped. I0327 15:10:03.816692 1 recorder.go:75] Recording config/ingress with fingerprint=acccaa995b110d787c7538e500450ef0766ec92a893e3f077d1e693733ea9a49 I0327 15:10:03.816714 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 27.547412ms to process 1 records I0327 15:10:03.816722 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 31.322466ms to process 0 records I0327 15:10:03.816730 1 tasks_processing.go:74] worker 4 stopped. I0327 15:10:03.817061 1 tasks_processing.go:74] worker 55 stopped. I0327 15:10:03.817459 1 recorder.go:75] Recording config/node/ip-10-0-0-192.ec2.internal with fingerprint=4f32378d595ab02b2d31c77fe39f1ad5c8b094b262a120ed31632d08b1fff1a5 I0327 15:10:03.817535 1 recorder.go:75] Recording config/node/ip-10-0-1-144.ec2.internal with fingerprint=6a5dbf765d05b93a619ec28cc718dfea528d8bd8ae52324d9743739c1266cc7a I0327 15:10:03.817626 1 recorder.go:75] Recording config/node/ip-10-0-2-34.ec2.internal with fingerprint=5b65fdba05f4e18f4ebcb84aec7105b11aca2459fe82fb21300ce19b58196edc I0327 15:10:03.817640 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 28.313756ms to process 3 records I0327 15:10:03.817660 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0327 15:10:03.817667 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 28.471787ms to process 1 records I0327 15:10:03.817716 1 tasks_processing.go:74] worker 48 stopped. I0327 15:10:03.817788 1 tasks_processing.go:74] worker 13 stopped. W0327 15:10:03.817927 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 15:10:03.818384 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=11f40bf79d0c109f91d9ac2b041aad47fb25e75b5deb16a5ea3d94c5d3811fa1 I0327 15:10:03.819573 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=15d569cf06778bbccac569c020da040f60eaadf61a274d8a175bf6b1861a2b91 I0327 15:10:03.819612 1 gather.go:177] gatherer "clusterconfig" function "crds" took 32.2081ms to process 2 records I0327 15:10:03.819753 1 tasks_processing.go:74] worker 42 stopped. I0327 15:10:03.819853 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=1cf6fc651a6b73baf8cbfecadcf3175336138bf5745cc96c238aabaafcc4756a I0327 15:10:03.819992 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=66edc62add237ff45449b34079f3b45794860f72794fe59e713779c85c02c9b4 I0327 15:10:03.820028 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=bd541d7ff1a4ada3a064a0a899f0389e83b8068052baa3e1d7afb28191283fd4 I0327 15:10:03.820075 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=8a8cc0088bf30cc4c96bbc7ce2b0cf9397acd1b48660be94e6402415e4d51a6e I0327 15:10:03.820127 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=eb858ec8ad5cab0773568582c8ebd34d52c8e0cf5247904aa4e0dd4740f5ffc3 I0327 15:10:03.820190 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=956f58d2a28d3599d67211b31be72a084e2ab3ec2b888813054cc3c0e597edb8 I0327 15:10:03.820233 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=ce2982d1644d9b4a0e7be5bffa3ea64845ff3be2a08e7736c2dc0b750837f1d6 I0327 15:10:03.820306 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=d0cdb37605ecd78378def9a05275d82b34ca17dcd389910eeb72aed0fcf271c0 I0327 15:10:03.820347 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=bb44811901c84278ca3461a4bd1cd00950639a1c198341979d97c4c04b4b2285 I0327 15:10:03.820397 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=94485e800c572e976ab664238a9b81c5ca0baf4949628bfe2c0478df57fe7c89 I0327 15:10:03.820441 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=086e83d79970ce09fbf4d32dca9ee357832a6b67f6fdc1dd69c3bb1e109c038a I0327 15:10:03.820461 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 29.00963ms to process 11 records I0327 15:10:03.820715 1 tasks_processing.go:74] worker 39 stopped. I0327 15:10:03.821796 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=c179253cc9272d7f8061b783717695a1cffe98d8b62c6f78f23cefb4b7ed6e4b I0327 15:10:03.821818 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 33.046085ms to process 1 records I0327 15:10:03.821828 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0327 15:10:03.821846 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0327 15:10:03.821891 1 tasks_processing.go:74] worker 50 stopped. I0327 15:10:03.821898 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e3238c1bfd55938c83c80b692905f51f056caa3f189e2083704ff86a2dfdd608 I0327 15:10:03.821920 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 31.600637ms to process 1 records W0327 15:10:03.821986 1 operator.go:328] started I0327 15:10:03.822009 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0327 15:10:03.822232 1 tasks_processing.go:74] worker 54 stopped. E0327 15:10:03.822246 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0327 15:10:03.822268 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0327 15:10:03.822284 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 36.184845ms to process 1 records I0327 15:10:03.824560 1 tasks_processing.go:74] worker 12 stopped. I0327 15:10:03.824737 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=141330a6d3c76644a8f1d9ac60604f3a2f5eeef86c4c4c5bb6de95afcf0503db I0327 15:10:03.824826 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=22c96d0d03508be0a5d0a501a792c3805c2907a57227671a1da54237f225652d I0327 15:10:03.824835 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 39.379928ms to process 2 records I0327 15:10:03.825487 1 tasks_processing.go:74] worker 33 stopped. I0327 15:10:03.825498 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 37.225682ms to process 0 records I0327 15:10:03.829106 1 controller.go:254] Source scaController *sca.Controller is not ready I0327 15:10:03.829123 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0327 15:10:03.829129 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0327 15:10:03.829133 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0327 15:10:03.829137 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0327 15:10:03.829162 1 controller.go:531] The operator is still being initialized I0327 15:10:03.829174 1 controller.go:554] The operator is healthy I0327 15:10:03.834209 1 prometheus_rules.go:88] Prometheus rules successfully created I0327 15:10:03.838386 1 tasks_processing.go:74] worker 47 stopped. I0327 15:10:03.838402 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 50.001105ms to process 0 records E0327 15:10:03.840231 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2738c646dd-6139-44f6-a856-5a3dc9e0c903%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:41716->172.30.0.10:53: read: connection refused I0327 15:10:03.840244 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2738c646dd-6139-44f6-a856-5a3dc9e0c903%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:41716->172.30.0.10:53: read: connection refused I0327 15:10:03.842356 1 tasks_processing.go:74] worker 19 stopped. E0327 15:10:03.842371 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0327 15:10:03.842377 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0327 15:10:03.842380 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0327 15:10:03.842391 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=7c953bef0cdbddb25a47a3c40772dfa7cbf73ae2ac9bb3dc97984dbf1bd43499 I0327 15:10:03.842425 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0327 15:10:03.842432 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0327 15:10:03.842437 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=4bec6f0bfa45f0ccf5709371166150dcb9cf2703d04f5f6de4476d0277c477a6 I0327 15:10:03.842441 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0327 15:10:03.842479 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0327 15:10:03.842486 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0327 15:10:03.842491 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 54.605761ms to process 7 records I0327 15:10:03.859370 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0327 15:10:03.859383 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0327 15:10:03.859405 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0327 15:10:03.863103 1 base_controller.go:82] Caches are synced for ConfigController I0327 15:10:03.863116 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0327 15:10:03.867022 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 15:10:03.904639 1 tasks_processing.go:74] worker 17 stopped. I0327 15:10:03.904964 1 recorder.go:75] Recording config/version with fingerprint=0bc4623f79aa7acaaaf29e8c174698c35abb631fccaf36bf7269ef4d2db5adb6 I0327 15:10:03.904982 1 recorder.go:75] Recording config/id with fingerprint=b8c9174460481be3ca338cba4fc14df3ec9ba1d387c28b16a5617b870b5dba80 I0327 15:10:03.904989 1 gather.go:177] gatherer "clusterconfig" function "version" took 115.874223ms to process 2 records I0327 15:10:03.913698 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0327 15:10:03.918372 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:34279->172.30.0.10:53: read: connection refused I0327 15:10:03.918426 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.14:34279->172.30.0.10:53: read: connection refused I0327 15:10:03.920122 1 tasks_processing.go:74] worker 7 stopped. I0327 15:10:03.921367 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-8777bd9bb-8xc4s with fingerprint=1002af0b33968ed091c7d33c35220bbd69e796c877f538e5727bc954ef9fac8c I0327 15:10:03.921657 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-f76cq with fingerprint=30fc98788be9a8f5d0db480c22f8ea5d9dc9be15f520f4ba11d7b8cdee3a58b7 I0327 15:10:03.921705 1 recorder.go:75] Recording config/running_containers with fingerprint=3e406fb1129a94f739b5826e09173e8e4e3a2da9fa7d2fd6b19bfec446c51a1f I0327 15:10:03.921715 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 133.343991ms to process 3 records I0327 15:10:03.922214 1 base_controller.go:82] Caches are synced for LoggingSyncer I0327 15:10:03.922226 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0327 15:10:03.935401 1 tasks_processing.go:74] worker 57 stopped. E0327 15:10:03.935418 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0327 15:10:03.935424 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pa4nkn99kaqqvukhcr451snphcdn29u-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pa4nkn99kaqqvukhcr451snphcdn29u-primary-cert-bundle-secret" not found I0327 15:10:03.935487 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=c542303b480faca948e493bd43dbef3894e4a1679ce71cc3617e73093e345006 I0327 15:10:03.935500 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 146.541043ms to process 1 records I0327 15:10:04.203404 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0327 15:10:04.203420 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0327 15:10:04.203640 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-csqt7 pod in namespace openshift-dns (previous: false). I0327 15:10:04.461344 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-csqt7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-csqt7\" is waiting to start: ContainerCreating" I0327 15:10:04.461363 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-csqt7\" is waiting to start: ContainerCreating" I0327 15:10:04.461372 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-csqt7 pod in namespace openshift-dns (previous: false). I0327 15:10:04.611793 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-csqt7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-csqt7\" is waiting to start: ContainerCreating" I0327 15:10:04.611812 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-csqt7\" is waiting to start: ContainerCreating" I0327 15:10:04.611823 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-kkz7f pod in namespace openshift-dns (previous: false). W0327 15:10:04.813744 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 15:10:04.832320 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kkz7f pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-kkz7f\" is waiting to start: ContainerCreating" I0327 15:10:04.832340 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-kkz7f\" is waiting to start: ContainerCreating" I0327 15:10:04.832347 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-kkz7f pod in namespace openshift-dns (previous: false). I0327 15:10:05.010396 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kkz7f pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-kkz7f\" is waiting to start: ContainerCreating" I0327 15:10:05.010416 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-kkz7f\" is waiting to start: ContainerCreating" I0327 15:10:05.010427 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-mrt9w pod in namespace openshift-dns (previous: false). I0327 15:10:05.225719 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mrt9w pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-mrt9w\" is waiting to start: ContainerCreating" I0327 15:10:05.225740 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-mrt9w\" is waiting to start: ContainerCreating" I0327 15:10:05.225748 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-mrt9w pod in namespace openshift-dns (previous: false). I0327 15:10:05.239668 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0327 15:10:05.411113 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mrt9w pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-mrt9w\" is waiting to start: ContainerCreating" I0327 15:10:05.411132 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-mrt9w\" is waiting to start: ContainerCreating" I0327 15:10:05.411143 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8fzl8 pod in namespace openshift-dns (previous: false). I0327 15:10:05.609104 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0327 15:10:05.609122 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-vkk6p pod in namespace openshift-dns (previous: false). I0327 15:10:05.811129 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0327 15:10:05.811150 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-vnpnx pod in namespace openshift-dns (previous: false). W0327 15:10:05.813864 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 15:10:05.842664 1 tasks_processing.go:74] worker 22 stopped. I0327 15:10:05.842710 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=c700193ce28c8f282b8182b2f337d9db26a7a9571f94977fd98e2c68af2313b2 I0327 15:10:05.842754 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=f09bc59af3d31aca7e7c025ca368d6017077c154f02088d806bb5c86d2050570 I0327 15:10:05.842783 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0327 15:10:05.842806 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=61f6d9d6e6e1a6f53932f80b2ec4d5d981c1a2eac0145da7fb72fb56fe6d7345 I0327 15:10:05.842838 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0327 15:10:05.842862 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=3aedee60f7c97cb68175e5166364603025e33fe0124687bb27488f873b90b57f I0327 15:10:05.842899 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=d0e7f43832712d6e9e145f4f49b5b2fbe8fe35721a727c6f622ab9fd57e021c0 I0327 15:10:05.842930 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=699a602c004add79a1103523404fffd7a388e8fc42e2af799ef21bda74bf530f I0327 15:10:05.842949 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=370c00715bc7d93a93970acd1e0abe2602f72a2261603e1b61dcf1c0b59493b6 I0327 15:10:05.842969 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=963fd6a724d01f690d9c308c89e8ec3d3f70d8e37399fceb697dcde964b127c5 I0327 15:10:05.842978 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0327 15:10:05.842994 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=37d4d479e80000627443ce767496796573e4f99ecf4ef213ac9d629216bbc1db I0327 15:10:05.843003 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0327 15:10:05.843019 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=2802c8955c2a09582d540a81cba1bff9e78fc07c3ff164aeb44ab80e4a86c715 I0327 15:10:05.843028 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0327 15:10:05.843047 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=21993b620b51bbdc6253dd61c50de2dc6b4f96f88748017c0602dcb9f0738d3d I0327 15:10:05.843056 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0327 15:10:05.843071 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=9e80d4999d893f6a7f03b248661194e55b3141a74d7b7b6ef6ba85212de64832 I0327 15:10:05.843181 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=3fce51921137a2c9ab2d332cf845ec1b51f065eb4fdff79c9210ec187cf8d787 I0327 15:10:05.843191 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0327 15:10:05.843199 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0327 15:10:05.843221 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0327 15:10:05.843243 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=296fa6bea21158ce371e50bdfcc3bfd7b019cee6093d47a25987c56e6851c133 I0327 15:10:05.843268 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=943a875cfd1ced5d00fe624546646989c2367d1ff5f9d516a291ac46ea3f3f15 I0327 15:10:05.843277 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0327 15:10:05.843293 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=95e73e3e0e1ae3a4f6d994912d4664920e1cf5aa417d88f3c1b7598ae66a17f0 I0327 15:10:05.843302 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0327 15:10:05.843314 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=826ba63b2967c25807da1209a7d50422b600a9cf3190f526a1b5453b715e7b64 I0327 15:10:05.843327 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=439effe2ca0629e4df7a7484f52baa42ae6042288f6a23e2526dec8265665cf0 I0327 15:10:05.843341 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=a4def06964ef5dccbbfb779805832f5b31112f653efdba6e19778daf5ac3f14c I0327 15:10:05.843356 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=00dcd5443603a86e1ada12e4b4ed5d107938b8e0b84b0370dc5e8b7297c75d9b I0327 15:10:05.843374 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=e5465aaa94875b7fb359fecd9ed6b2d0c0b598d0810c8699a06d888130be9b37 I0327 15:10:05.843383 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0327 15:10:05.843411 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=5b967715e8a8faf62cb811e6f245ca8c459ba11b13a59c3b07302bf586798845 I0327 15:10:05.843428 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0327 15:10:05.843434 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0327 15:10:05.843442 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.056997578s to process 36 records I0327 15:10:06.008731 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0327 15:10:06.008748 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-59c85fb9b8-2rcd8 pod in namespace openshift-image-registry (previous: false). I0327 15:10:06.211572 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-59c85fb9b8-2rcd8 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-59c85fb9b8-2rcd8\" is waiting to start: ContainerCreating" I0327 15:10:06.211606 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-59c85fb9b8-2rcd8\" is waiting to start: ContainerCreating" I0327 15:10:06.211620 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7c85498c95-7sbxq pod in namespace openshift-image-registry (previous: false). I0327 15:10:06.408105 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7c85498c95-7sbxq pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7c85498c95-7sbxq\" is waiting to start: ContainerCreating" I0327 15:10:06.408125 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7c85498c95-7sbxq\" is waiting to start: ContainerCreating" I0327 15:10:06.408137 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7c85498c95-q75rf pod in namespace openshift-image-registry (previous: false). I0327 15:10:06.610911 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7c85498c95-q75rf pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7c85498c95-q75rf\" is waiting to start: ContainerCreating" I0327 15:10:06.610931 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7c85498c95-q75rf\" is waiting to start: ContainerCreating" I0327 15:10:06.610941 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9gb24 pod in namespace openshift-image-registry (previous: false). I0327 15:10:06.809388 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0327 15:10:06.809406 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-hbzjk pod in namespace openshift-image-registry (previous: false). W0327 15:10:06.813917 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 15:10:07.014316 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0327 15:10:07.014337 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-rnf7v pod in namespace openshift-image-registry (previous: false). I0327 15:10:07.209230 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0327 15:10:07.209249 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-859d5bccd5-cr7vc pod in namespace openshift-ingress (previous: false). I0327 15:10:07.410973 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-859d5bccd5-cr7vc pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-859d5bccd5-cr7vc\" is waiting to start: ContainerCreating" I0327 15:10:07.410992 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-859d5bccd5-cr7vc\" is waiting to start: ContainerCreating" I0327 15:10:07.411007 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-f9877767d-69sb2 pod in namespace openshift-ingress (previous: false). I0327 15:10:07.608966 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-f9877767d-69sb2 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-f9877767d-69sb2\" is waiting to start: ContainerCreating" I0327 15:10:07.608984 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-f9877767d-69sb2\" is waiting to start: ContainerCreating" I0327 15:10:07.609006 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-f9877767d-9bxmh pod in namespace openshift-ingress (previous: false). I0327 15:10:07.811194 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-f9877767d-9bxmh pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-f9877767d-9bxmh\" is waiting to start: ContainerCreating" I0327 15:10:07.811214 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-f9877767d-9bxmh\" is waiting to start: ContainerCreating" I0327 15:10:07.811224 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-db7t2 pod in namespace openshift-ingress-canary (previous: false). W0327 15:10:07.813722 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0327 15:10:08.007749 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-qg9vn pod in namespace openshift-ingress-canary (previous: false). I0327 15:10:08.211380 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-xpx9l pod in namespace openshift-ingress-canary (previous: false). I0327 15:10:08.408780 1 tasks_processing.go:74] worker 23 stopped. I0327 15:10:08.408877 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=eeaad08099ab13864e19d7ad89f1bc5fffd92755c6a9893139985ebfd9233b14 I0327 15:10:08.408935 1 recorder.go:75] Recording events/openshift-dns with fingerprint=d0ef53832531c36917306871fcaec842a5c81f462d61c03fdef16ebe5b67ee18 I0327 15:10:08.409032 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=ee93bf523da16f8a5d7a97812cd9b3974bef636eee32bc431066a52a06239ba6 I0327 15:10:08.409060 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=c22b0aafb3bc016f77fd7708134773cf28a87e4db888ccc89963386c06b42000 I0327 15:10:08.409105 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=5322de2757b1b3bbd0a8bb8b7f82fc13e3910877a121828dc0b8812628f41c7a I0327 15:10:08.409143 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=cc91a1eddb45ad28cb5342ce38afaa83fb75ef05a17e7165764f95583657f505 I0327 15:10:08.409150 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-db7t2/serve-healthcheck-canary_current.log with fingerprint=61927a8ec421e597ad782142ceb376cbe315396b5958e776b6a157012ccac4b0 I0327 15:10:08.409155 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-qg9vn/serve-healthcheck-canary_current.log with fingerprint=11a7ec25cc0e5a25475287d4b4b06bf18e1c16d81438409870938d151a11a46b I0327 15:10:08.409160 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-xpx9l/serve-healthcheck-canary_current.log with fingerprint=baf1d641c245e8f7c70b99df234103ef48268346353a608b000fd3cb8d099474 I0327 15:10:08.409166 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.622902225s to process 9 records W0327 15:10:08.809932 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0327 15:10:08.809968 1 tasks_processing.go:74] worker 36 stopped. E0327 15:10:08.809983 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0327 15:10:08.809995 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0327 15:10:08.810010 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0327 15:10:08.810020 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.024651433s to process 1 records I0327 15:10:16.384543 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 15:10:16.433807 1 tasks_processing.go:74] worker 25 stopped. I0327 15:10:16.433846 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0327 15:10:16.433859 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.645581511s to process 1 records I0327 15:10:17.195233 1 tasks_processing.go:74] worker 46 stopped. I0327 15:10:17.195514 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=7ddf795073bc5395a95429a28409923bc71b328da94f586bee093d6f578708e6 I0327 15:10:17.195533 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.406968094s to process 1 records E0327 15:10:17.195610 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.41s with: function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0327 15:10:17.196723 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0327 15:10:17.196741 1 periodic.go:216] Running workloads gatherer I0327 15:10:17.196760 1 tasks_processing.go:45] number of workers: 2 I0327 15:10:17.196773 1 tasks_processing.go:69] worker 1 listening for tasks. I0327 15:10:17.196777 1 tasks_processing.go:71] worker 1 working on workload_info task. I0327 15:10:17.196785 1 tasks_processing.go:69] worker 0 listening for tasks. I0327 15:10:17.196802 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0327 15:10:17.219508 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0327 15:10:17.227875 1 gather_workloads_info.go:387] No image sha256:273930332d7b481bbca4521cf740507498a2a323a4010cd66e896806b299f85e (9ms) I0327 15:10:17.231605 1 tasks_processing.go:74] worker 0 stopped. I0327 15:10:17.231621 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 34.767209ms to process 0 records I0327 15:10:17.234819 1 gather_workloads_info.go:387] No image sha256:0fd69333c121efd68919e731cd3ea01e976e00415bda6e987bf1e949ec224eda (7ms) I0327 15:10:17.241716 1 gather_workloads_info.go:387] No image sha256:72ae38682f25786aec18625cfed5d7831017d2f0f1304c3320c00ce0b7fb09c1 (7ms) I0327 15:10:17.248261 1 gather_workloads_info.go:387] No image sha256:83344c7d1374cbc2607b4fcb837f094c035b32daf24f8b3ccba2c8df409a6d3c (7ms) I0327 15:10:17.254806 1 gather_workloads_info.go:387] No image sha256:aa65d1c7fb8805b767fc4878faea280472f372010efadc66e5641a2e634a80c0 (7ms) I0327 15:10:17.261780 1 gather_workloads_info.go:387] No image sha256:4c39dc633bf3560c2397281b74faf9d00500b0738dec6c1aad2cc8ce326be6fc (7ms) I0327 15:10:17.269339 1 gather_workloads_info.go:387] No image sha256:2f26472542da4f89a2ac34a93fd438d385419dda16811bdfa7e679e0115abe56 (8ms) I0327 15:10:17.275765 1 gather_workloads_info.go:387] No image sha256:8f72efd51f564d8d3e4ecee6555d7bd5394d0776054d2112febea6b7c9e71225 (6ms) I0327 15:10:17.285092 1 gather_workloads_info.go:387] No image sha256:4e62a2dcb8214491388d4c17f89fe699eb53cdb8039dcba6ab052edc28da3fbd (9ms) I0327 15:10:17.292065 1 gather_workloads_info.go:387] No image sha256:a20df9dd8819885aa431022c6f5e80b77b4431f268b719d71bec26f10521da0e (7ms) I0327 15:10:17.329614 1 gather_workloads_info.go:387] No image sha256:12aec97ddeb527881acf5949bea2b227baa1dbfccf30679e8d57ff0b82e42707 (38ms) I0327 15:10:17.428078 1 gather_workloads_info.go:387] No image sha256:6ea0f19030ffc52a290e6eeb812ef2426c48a2f1083b1e415e0667998cbb7bf2 (98ms) I0327 15:10:17.527647 1 gather_workloads_info.go:387] No image sha256:3da5a449ace61cb7d002d8e7911c14905c74cf58a296527458d1af0f1abf0a72 (100ms) I0327 15:10:17.627777 1 gather_workloads_info.go:387] No image sha256:ad0e262327997e8cdc4074a2373e67274d9b8a943aa274a2cd8ce98ffe1db4a0 (100ms) I0327 15:10:17.726990 1 gather_workloads_info.go:387] No image sha256:1e6d59867cbd750672bc9ccfb9c69c3d198174cd43dd6d71a0da358b4b766473 (99ms) I0327 15:10:17.826662 1 gather_workloads_info.go:387] No image sha256:185a95b130b6a27ad88fcf39803215b7bd998472b1e37225858435f4eb7c3c04 (100ms) I0327 15:10:17.929222 1 gather_workloads_info.go:387] No image sha256:3739c0c55f7f7ccc84309ae11eb23d3d9e5ad50a68e849a8a8dfda7f2f2bc137 (103ms) I0327 15:10:18.027355 1 gather_workloads_info.go:387] No image sha256:66640a3f7bd65090345552611701a73e74bc44897b9d29e43b11f73b96598ac0 (98ms) I0327 15:10:18.129161 1 gather_workloads_info.go:387] No image sha256:2ade6dfb5406ef3a4264dea95d455298f54e8ef4d458d9ee3e94c33d48676a9b (102ms) I0327 15:10:18.226907 1 gather_workloads_info.go:387] No image sha256:20a3e59566dd25058fff22e18569067fe6a12d8ac5ac62fa6508e0cee44d7897 (98ms) I0327 15:10:18.327457 1 gather_workloads_info.go:387] No image sha256:12448aad6e9050c2b2445035be818a490cef502d39b73dbd51cc0f5ea7d67d40 (101ms) I0327 15:10:18.427238 1 gather_workloads_info.go:387] No image sha256:87de118dc29ca1b47d4768ec5ed4441bc1dada3fb2a78d53b1833249baa86dd4 (100ms) I0327 15:10:18.526894 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (100ms) I0327 15:10:18.626950 1 gather_workloads_info.go:387] No image sha256:1b58ba57c10e2ba03a0d8dba1f3f8faa866e1dca6170725b6909c2cc2b96efa6 (100ms) I0327 15:10:18.727341 1 gather_workloads_info.go:387] No image sha256:a6b085cdad58db8529d99c22ec1ec79fe09ab00ab1c92ee38b29e7875dc0533c (100ms) I0327 15:10:18.827090 1 gather_workloads_info.go:387] No image sha256:cfcb0bd30c537e677ac6856c36c0c923b415f24d944c7aa1386489f511f6491d (100ms) I0327 15:10:18.926760 1 gather_workloads_info.go:387] No image sha256:6263faa8ef7759ec21f26df08314c5d64e52d967b49961cb9404c54532c03365 (100ms) I0327 15:10:19.026617 1 gather_workloads_info.go:387] No image sha256:1c5b548d67e5c56e3297b2f8aa34428e544b5b5ac88bd8d0f465760375c501e6 (100ms) I0327 15:10:19.127676 1 gather_workloads_info.go:387] No image sha256:51e1b5dc9fccbee5b7df78d0e8bbe379f75529232c102c9ed5b1ae134c4b9b4c (101ms) I0327 15:10:19.226559 1 gather_workloads_info.go:387] No image sha256:2e3661304917ce7f3be49974faacc60c24cbe2833d7b50c8c205c2f3f2467950 (99ms) I0327 15:10:19.326758 1 gather_workloads_info.go:387] No image sha256:31e175b644c6627779ca6b9f48d9426760dd449788ad671dd0277cce8fb822e8 (100ms) I0327 15:10:19.387054 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 15:10:19.426698 1 gather_workloads_info.go:387] No image sha256:5b51a4d9d3b7ecad110fb6eb00f71e9e9b854067d79b220df397a58d9fa50eb0 (100ms) I0327 15:10:19.426733 1 tasks_processing.go:74] worker 1 stopped. E0327 15:10:19.426743 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0327 15:10:19.426992 1 recorder.go:75] Recording config/workload_info with fingerprint=e73af24dd3b6a1d2e6b491f6fd7044812e3699c8900e04a52fc230102ac8e498 I0327 15:10:19.427007 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.229944742s to process 1 records E0327 15:10:19.427031 1 periodic.go:254] "Unhandled Error" err="workloads failed after 2.23s with: function \"workload_info\" failed with an error" I0327 15:10:19.428133 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0327 15:10:19.428146 1 periodic.go:216] Running conditional gatherer I0327 15:10:19.432683 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.6/gathering_rules I0327 15:10:19.439596 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.6/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.14:60785->172.30.0.10:53: read: connection refused E0327 15:10:19.439905 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0327 15:10:19.439967 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0327 15:10:19.444730 1 conditional_gatherer.go:392] cluster version is '4.21.6' E0327 15:10:19.444743 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444749 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444752 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444756 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444759 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444762 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444764 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444767 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0327 15:10:19.444770 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0327 15:10:19.444783 1 tasks_processing.go:45] number of workers: 3 I0327 15:10:19.444795 1 tasks_processing.go:69] worker 2 listening for tasks. I0327 15:10:19.444799 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0327 15:10:19.444804 1 tasks_processing.go:69] worker 0 listening for tasks. I0327 15:10:19.444815 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0327 15:10:19.444818 1 tasks_processing.go:69] worker 1 listening for tasks. I0327 15:10:19.444823 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0327 15:10:19.444826 1 tasks_processing.go:74] worker 1 stopped. I0327 15:10:19.444881 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0327 15:10:19.444894 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 978ns to process 1 records I0327 15:10:19.444925 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0327 15:10:19.444933 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.162µs to process 1 records I0327 15:10:19.444939 1 tasks_processing.go:74] worker 0 stopped. I0327 15:10:19.445065 1 tasks_processing.go:74] worker 2 stopped. I0327 15:10:19.445079 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 233.085µs to process 0 records I0327 15:10:19.445098 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.6/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.14:60785->172.30.0.10:53: read: connection refused I0327 15:10:19.445115 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0327 15:10:19.464923 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=b1f2e405e5676d2b1b67745efe8cd951b152817da23c0226a141cca4476c2a22 I0327 15:10:19.465043 1 diskrecorder.go:70] Writing 105 records to /var/lib/insights-operator/insights-2026-03-27-151019.tar.gz I0327 15:10:19.471406 1 diskrecorder.go:51] Wrote 105 records to disk in 6ms I0327 15:10:19.471435 1 periodic.go:285] Gathering cluster info every 2h0m0s I0327 15:10:19.471449 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0327 15:10:19.586551 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 15:10:30.416571 1 configmapobserver.go:84] configmaps "insights-config" not found I0327 15:11:12.767918 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="ccea4565015245424317eb7ef8882f64aad7d189a828d372128f0a212c6370de") W0327 15:11:12.767957 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0327 15:11:12.768014 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="0b700e6bcbea97a554b24e7cc1fe84529941c6092433cd09557f8b24954c1ca9")