W0311 08:23:14.388731 1 cmd.go:257] Using insecure, self-signed certificates I0311 08:23:14.542891 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 08:23:14.543207 1 observer_polling.go:159] Starting file observer W0311 08:23:14.563290 1 builder.go:272] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-5ff5cb4f99" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0311 08:23:15.317107 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0311 08:23:15.317333 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0311 08:23:15.317900 1 secure_serving.go:57] Forcing use of http/1.1 only W0311 08:23:15.317920 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0311 08:23:15.317927 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0311 08:23:15.317930 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0311 08:23:15.317933 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0311 08:23:15.317936 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0311 08:23:15.317939 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0311 08:23:15.317961 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0311 08:23:15.320831 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes CPMSMachineNamePrefix ChunkSizeMiB CloudDualStackNodeIPs ConsolePluginContentSecurityPolicy DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags GatewayAPI GatewayAPIController HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy OnClusterBuild PersistentIPsForVirtualization PrivateHostedZoneAWS RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController VSphereDriverConfiguration VSphereMultiVCenters ValidatingAdmissionPolicy AWSClusterHostedDNS AutomatedEtcdBackup BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNS GCPCustomAPIEndpoints HighlyAvailableArbiter ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements SELinuxChangePolicy SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerification SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0311 08:23:15.320909 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ChunkSizeMiB", "CloudDualStackNodeIPs", "ConsolePluginContentSecurityPolicy", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "GatewayAPI", "GatewayAPIController", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "OnClusterBuild", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "VSphereDriverConfiguration", "VSphereMultiVCenters", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AutomatedEtcdBackup", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "HighlyAvailableArbiter", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "SELinuxChangePolicy", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0311 08:23:15.321690 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0311 08:23:15.321705 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0311 08:23:15.321707 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0311 08:23:15.321715 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0311 08:23:15.321726 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0311 08:23:15.321729 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0311 08:23:15.322006 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-4268190929/tls.crt::/tmp/serving-cert-4268190929/tls.key" I0311 08:23:15.322706 1 secure_serving.go:213] Serving securely on [::]:8443 I0311 08:23:15.322800 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" E0311 08:23:15.324296 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.189bbbd37da3ca51 openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdditionalRoutingCapabilities\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"CPMSMachineNamePrefix\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"ConsolePluginContentSecurityPolicy\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"GatewayAPI\", \"GatewayAPIController\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MetricsCollectionProfiles\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"OnClusterBuild\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SetEIPForNLBIngressController\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ClusterVersionOperatorConfiguration\", \"DNSNameResolver\", \"DualReplica\", \"DyanmicServiceEndpointIBMCloud\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"Example2\", \"ExternalOIDCWithUIDAndExtraClaimMappings\", \"GCPClusterHostedDNS\", \"GCPCustomAPIEndpoints\", \"HighlyAvailableArbiter\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"KMSEncryptionProvider\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NewOLMCatalogdAPIV1Metas\", \"NewOLMOwnSingleNamespace\", \"NewOLMPreflightPermissionChecks\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"SELinuxChangePolicy\", \"SELinuxMount\", \"ShortCertRotation\", \"SignatureStores\", \"SigstoreImageVerification\", \"SigstoreImageVerificationPKI\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereConfigurableMaxAllowedBlockVolumesPerNode\", \"VSphereHostVMGroupZonal\", \"VSphereMultiDisk\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-03-11 08:23:15.320801873 +0000 UTC m=+0.971798220,LastTimestamp:2026-03-11 08:23:15.320801873 +0000 UTC m=+0.971798220,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W0311 08:23:15.325951 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0311 08:23:15.325978 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0311 08:23:15.326076 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0311 08:23:15.329732 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0311 08:23:15.329756 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0311 08:23:15.333209 1 secretconfigobserver.go:119] support secret does not exist I0311 08:23:15.336479 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0311 08:23:15.339348 1 secretconfigobserver.go:119] support secret does not exist I0311 08:23:15.342106 1 recorder.go:156] Pruning old reports every 7h10m18s, max age is 288h0m0s I0311 08:23:15.345808 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0311 08:23:15.345820 1 periodic.go:212] Running clusterconfig gatherer I0311 08:23:15.345828 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0311 08:23:15.345831 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0311 08:23:15.345838 1 insightsreport.go:296] Starting report retriever I0311 08:23:15.345846 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0311 08:23:15.345877 1 tasks_processing.go:45] number of workers: 32 I0311 08:23:15.345909 1 tasks_processing.go:69] worker 31 listening for tasks. I0311 08:23:15.345916 1 tasks_processing.go:71] worker 31 working on service_accounts task. I0311 08:23:15.345918 1 tasks_processing.go:69] worker 14 listening for tasks. I0311 08:23:15.345927 1 tasks_processing.go:69] worker 6 listening for tasks. I0311 08:23:15.345930 1 tasks_processing.go:69] worker 7 listening for tasks. I0311 08:23:15.345934 1 tasks_processing.go:69] worker 0 listening for tasks. I0311 08:23:15.345937 1 tasks_processing.go:69] worker 22 listening for tasks. I0311 08:23:15.345939 1 tasks_processing.go:69] worker 8 listening for tasks. I0311 08:23:15.345943 1 tasks_processing.go:69] worker 1 listening for tasks. I0311 08:23:15.345945 1 tasks_processing.go:69] worker 19 listening for tasks. I0311 08:23:15.345951 1 tasks_processing.go:69] worker 9 listening for tasks. I0311 08:23:15.345953 1 tasks_processing.go:69] worker 20 listening for tasks. I0311 08:23:15.345956 1 tasks_processing.go:69] worker 2 listening for tasks. I0311 08:23:15.345959 1 tasks_processing.go:69] worker 21 listening for tasks. I0311 08:23:15.345960 1 tasks_processing.go:69] worker 10 listening for tasks. I0311 08:23:15.345964 1 tasks_processing.go:69] worker 5 listening for tasks. I0311 08:23:15.345959 1 tasks_processing.go:69] worker 18 listening for tasks. I0311 08:23:15.345970 1 tasks_processing.go:69] worker 11 listening for tasks. I0311 08:23:15.345973 1 tasks_processing.go:69] worker 3 listening for tasks. I0311 08:23:15.345974 1 tasks_processing.go:69] worker 15 listening for tasks. I0311 08:23:15.345978 1 tasks_processing.go:69] worker 17 listening for tasks. I0311 08:23:15.345979 1 tasks_processing.go:69] worker 12 listening for tasks. I0311 08:23:15.345967 1 tasks_processing.go:69] worker 16 listening for tasks. I0311 08:23:15.345985 1 tasks_processing.go:69] worker 13 listening for tasks. I0311 08:23:15.345989 1 tasks_processing.go:69] worker 28 listening for tasks. I0311 08:23:15.345979 1 tasks_processing.go:69] worker 4 listening for tasks. I0311 08:23:15.345994 1 tasks_processing.go:69] worker 27 listening for tasks. I0311 08:23:15.345998 1 tasks_processing.go:69] worker 24 listening for tasks. I0311 08:23:15.345999 1 tasks_processing.go:69] worker 25 listening for tasks. I0311 08:23:15.345997 1 tasks_processing.go:69] worker 30 listening for tasks. I0311 08:23:15.346005 1 tasks_processing.go:69] worker 23 listening for tasks. I0311 08:23:15.345991 1 tasks_processing.go:69] worker 26 listening for tasks. I0311 08:23:15.346007 1 tasks_processing.go:69] worker 29 listening for tasks. I0311 08:23:15.346071 1 tasks_processing.go:71] worker 6 working on config_maps task. I0311 08:23:15.346078 1 tasks_processing.go:71] worker 1 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0311 08:23:15.346083 1 tasks_processing.go:71] worker 21 working on container_images task. I0311 08:23:15.346090 1 tasks_processing.go:71] worker 3 working on pdbs task. I0311 08:23:15.346095 1 tasks_processing.go:71] worker 17 working on monitoring_persistent_volumes task. I0311 08:23:15.346101 1 tasks_processing.go:71] worker 0 working on support_secret task. I0311 08:23:15.346072 1 tasks_processing.go:71] worker 14 working on proxies task. I0311 08:23:15.346161 1 tasks_processing.go:71] worker 12 working on mutating_webhook_configurations task. I0311 08:23:15.346165 1 tasks_processing.go:71] worker 8 working on storage_cluster task. I0311 08:23:15.346335 1 tasks_processing.go:71] worker 20 working on machine_healthchecks task. I0311 08:23:15.346394 1 tasks_processing.go:71] worker 27 working on openstack_dataplanenodesets task. I0311 08:23:15.346436 1 tasks_processing.go:71] worker 18 working on machines task. I0311 08:23:15.346451 1 tasks_processing.go:71] worker 11 working on infrastructures task. I0311 08:23:15.346548 1 tasks_processing.go:71] worker 7 working on node_logs task. I0311 08:23:15.346085 1 tasks_processing.go:71] worker 29 working on openstack_controlplanes task. I0311 08:23:15.346629 1 tasks_processing.go:71] worker 28 working on ceph_cluster task. I0311 08:23:15.346633 1 tasks_processing.go:71] worker 19 working on sap_config task. I0311 08:23:15.346737 1 tasks_processing.go:71] worker 4 working on image_pruners task. I0311 08:23:15.346161 1 tasks_processing.go:71] worker 22 working on pod_network_connectivity_checks task. I0311 08:23:15.346090 1 tasks_processing.go:71] worker 15 working on openstack_dataplanedeployments task. I0311 08:23:15.346405 1 tasks_processing.go:71] worker 30 working on nodenetworkconfigurationpolicies task. I0311 08:23:15.346410 1 tasks_processing.go:71] worker 24 working on openstack_version task. I0311 08:23:15.346415 1 tasks_processing.go:71] worker 25 working on networks task. I0311 08:23:15.346425 1 tasks_processing.go:71] worker 2 working on aggregated_monitoring_cr_names task. I0311 08:23:15.346608 1 tasks_processing.go:71] worker 10 working on machine_configs task. I0311 08:23:15.346606 1 tasks_processing.go:71] worker 23 working on ingress task. I0311 08:23:15.346616 1 tasks_processing.go:71] worker 13 working on operators task. I0311 08:23:15.346613 1 tasks_processing.go:71] worker 5 working on operators_pods_and_events task. I0311 08:23:15.346618 1 tasks_processing.go:71] worker 26 working on machine_sets task. I0311 08:23:15.346620 1 tasks_processing.go:71] worker 16 working on nodes task. I0311 08:23:15.349880 1 tasks_processing.go:71] worker 8 working on schedulers task. I0311 08:23:15.346628 1 tasks_processing.go:71] worker 9 working on ingress_certificates task. I0311 08:23:15.349998 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 3.69473ms to process 0 records I0311 08:23:15.350041 1 tasks_processing.go:71] worker 20 working on version task. E0311 08:23:15.350078 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0311 08:23:15.350440 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 3.690792ms to process 0 records I0311 08:23:15.351715 1 tasks_processing.go:71] worker 27 working on dvo_metrics task. I0311 08:23:15.351771 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 5.273661ms to process 0 records E0311 08:23:15.351800 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0311 08:23:15.351813 1 gather.go:177] gatherer "clusterconfig" function "machines" took 5.347749ms to process 0 records I0311 08:23:15.351842 1 tasks_processing.go:71] worker 18 working on machine_config_pools task. I0311 08:23:15.351872 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0311 08:23:15.351890 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0311 08:23:15.351901 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0311 08:23:15.351907 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0311 08:23:15.351920 1 controller.go:458] The operator is still being initialized I0311 08:23:15.351925 1 controller.go:481] The operator is healthy I0311 08:23:15.363092 1 tasks_processing.go:71] worker 14 working on storage_classes task. I0311 08:23:15.363284 1 recorder.go:70] Recording config/proxy with fingerprint=24e3b1f6bc97d0200a141fa16bfa51b626ee4509506129d3c6d7f056d307e8fb I0311 08:23:15.363308 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 16.922727ms to process 1 records I0311 08:23:15.363321 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 15.337842ms to process 0 records I0311 08:23:15.363330 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 15.462155ms to process 0 records I0311 08:23:15.363344 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 16.582458ms to process 0 records I0311 08:23:15.363353 1 tasks_processing.go:71] worker 28 working on clusterroles task. I0311 08:23:15.363396 1 tasks_processing.go:71] worker 15 working on silenced_alerts task. I0311 08:23:15.363447 1 tasks_processing.go:71] worker 22 working on certificate_signing_requests task. W0311 08:23:15.363473 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 08:23:15.363397 1 tasks_processing.go:71] worker 30 working on crds task. E0311 08:23:15.363586 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0311 08:23:15.363644 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 15.935203ms to process 0 records I0311 08:23:15.363856 1 tasks_processing.go:71] worker 11 working on lokistack task. I0311 08:23:15.364595 1 recorder.go:70] Recording config/infrastructure with fingerprint=12abd2531a203c4bfb69b9ae724171c8923eaf9c4c62bcc2c2303080eba152e6 I0311 08:23:15.364616 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 17.048919ms to process 1 records I0311 08:23:15.364632 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 67.763µs to process 0 records I0311 08:23:15.364640 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 15.37778ms to process 0 records I0311 08:23:15.364648 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 17.218402ms to process 0 records E0311 08:23:15.364661 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0311 08:23:15.364668 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 17.880174ms to process 0 records I0311 08:23:15.364673 1 tasks_processing.go:71] worker 24 working on install_plans task. I0311 08:23:15.364690 1 tasks_processing.go:71] worker 0 working on image task. I0311 08:23:15.364695 1 tasks_processing.go:71] worker 19 working on nodenetworkstates task. I0311 08:23:15.364699 1 tasks_processing.go:71] worker 29 working on cluster_apiserver task. I0311 08:23:15.364722 1 tasks_processing.go:71] worker 15 working on olm_operators task. I0311 08:23:15.364676 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 17.391259ms to process 0 records I0311 08:23:15.365017 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 14.54424ms to process 0 records I0311 08:23:15.365027 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 17.487575ms to process 0 records I0311 08:23:15.365098 1 tasks_processing.go:71] worker 26 working on machine_autoscalers task. I0311 08:23:15.365165 1 tasks_processing.go:71] worker 12 working on sap_datahubs task. I0311 08:23:15.365116 1 tasks_processing.go:71] worker 7 working on validating_webhook_configurations task. I0311 08:23:15.365271 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=612f4f3e3d2677b4207a9b84072ab5ae30a7808eb8d3a1f1ef2c7e85ea2be504 I0311 08:23:15.365428 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=09903ffc4a894b105062ebb43586e840054b6f5b2d85a2d04f3d432d118bb437 I0311 08:23:15.365461 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=1b4ac1729ee157e4a86a50e253521bef3043d01263359d959588aedff7537410 I0311 08:23:15.365466 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 17.992017ms to process 3 records I0311 08:23:15.365545 1 tasks_processing.go:71] worker 23 working on overlapping_namespace_uids task. I0311 08:23:15.365667 1 recorder.go:70] Recording config/ingress with fingerprint=f2ee13c257fb202f439b1aeaeda2f717b7aee5189779705ac678e65bdd561b7d I0311 08:23:15.365682 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 15.273139ms to process 1 records I0311 08:23:15.365773 1 tasks_processing.go:71] worker 3 working on active_alerts task. I0311 08:23:15.365778 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=03413de008d53f9cafdaeb87fec3a91feaa931cca25a89763e9a71f6f9780f18 W0311 08:23:15.365812 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 08:23:15.365816 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=c466ace748471bc6b23e3d72be085dddd1c7f417477a3a683d76e8bd6b66a361 I0311 08:23:15.365842 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=15a4c6bdbba179187b53deb156eeaa1a6801cec1efc4d9eb7d31e478f69f865c I0311 08:23:15.365853 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 18.30217ms to process 3 records I0311 08:23:15.365936 1 tasks_processing.go:71] worker 4 working on cost_management_metrics_configs task. I0311 08:23:15.366099 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=9eeb79da4fe1efec8cf8e3528fbf312bf5f30e97231bfbe5b4ecd797efa1831a I0311 08:23:15.366111 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 17.940178ms to process 1 records I0311 08:23:15.366117 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 36.851µs to process 0 records I0311 08:23:15.366124 1 tasks_processing.go:71] worker 3 working on sap_pods task. I0311 08:23:15.371404 1 tasks_processing.go:71] worker 8 working on authentication task. I0311 08:23:15.371545 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=cc755579865e0a3235cfce186a50db93d57deceb4781f689eebb79c1db2cb2bc I0311 08:23:15.371571 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 21.50037ms to process 1 records I0311 08:23:15.371583 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 25.324644ms to process 0 records I0311 08:23:15.371696 1 tasks_processing.go:71] worker 17 working on image_registries task. I0311 08:23:15.371804 1 recorder.go:70] Recording config/network with fingerprint=c6a9fa48c80e89738a82d9e842adf58995cfd8f90e10d05e23630197234e1cb8 I0311 08:23:15.371820 1 gather.go:177] gatherer "clusterconfig" function "networks" took 23.105401ms to process 1 records I0311 08:23:15.371881 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 20.027421ms to process 0 records I0311 08:23:15.371992 1 tasks_processing.go:71] worker 18 working on metrics task. I0311 08:23:15.372036 1 tasks_processing.go:71] worker 10 working on container_runtime_configs task. W0311 08:23:15.372049 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 08:23:15.372003 1 tasks_processing.go:71] worker 25 working on openshift_machine_api_events task. E0311 08:23:15.372062 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0311 08:23:15.372305 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0311 08:23:15.372340 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 23.281969ms to process 1 records I0311 08:23:15.372385 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 129.37µs to process 0 records I0311 08:23:15.372398 1 tasks_processing.go:71] worker 18 working on openshift_logging task. I0311 08:23:15.377170 1 tasks_processing.go:71] worker 21 working on tsdb_status task. W0311 08:23:15.377277 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 08:23:15.377380 1 recorder.go:70] Recording config/running_containers with fingerprint=e566ce4fc652344b387a8fc384296b13711f8adce68e325178059844bea892d6 I0311 08:23:15.377429 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 31.072593ms to process 1 records I0311 08:23:15.377444 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 65.076µs to process 0 records I0311 08:23:15.377453 1 tasks_processing.go:71] worker 21 working on oauths task. I0311 08:23:15.377909 1 tasks_processing.go:71] worker 4 working on feature_gates task. I0311 08:23:15.377999 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 11.957795ms to process 0 records I0311 08:23:15.378102 1 tasks_processing.go:71] worker 14 working on jaegers task. I0311 08:23:15.378516 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=5b866cb346d5cc3697d206d36ac1c6d049e1b16801373da2a0052771a9532c3c I0311 08:23:15.378547 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=dca929eb30510871b4efea277fe9573bb23823aad4d835ecc17a35c11932ead7 I0311 08:23:15.378557 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 14.882939ms to process 2 records I0311 08:23:15.379141 1 tasks_processing.go:74] worker 3 stopped. I0311 08:23:15.379165 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 13.002968ms to process 0 records I0311 08:23:15.379174 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 15.287468ms to process 0 records I0311 08:23:15.379181 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 14.465677ms to process 0 records I0311 08:23:15.379188 1 tasks_processing.go:74] worker 19 stopped. I0311 08:23:15.379193 1 tasks_processing.go:74] worker 11 stopped. W0311 08:23:15.382656 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 08:23:15.383053 1 tasks_processing.go:74] worker 0 stopped. I0311 08:23:15.383132 1 recorder.go:70] Recording config/image with fingerprint=6001d075c01608bfd48d404a716ceba33451491501e4edf689f93ac6f16c4152 I0311 08:23:15.383144 1 gather.go:177] gatherer "clusterconfig" function "image" took 18.35003ms to process 1 records I0311 08:23:15.383244 1 tasks_processing.go:74] worker 29 stopped. I0311 08:23:15.383277 1 recorder.go:70] Recording config/apiserver with fingerprint=8aeb384ad9a7759a1cd599996477784737748cb2b760c203dc2d3b89be073b2a I0311 08:23:15.383284 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 18.437029ms to process 1 records I0311 08:23:15.389839 1 tasks_processing.go:74] worker 14 stopped. I0311 08:23:15.389903 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 11.713249ms to process 0 records I0311 08:23:15.389942 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 17.740434ms to process 0 records I0311 08:23:15.389974 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 17.519922ms to process 0 records I0311 08:23:15.389987 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 24.716549ms to process 0 records I0311 08:23:15.389994 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 24.794255ms to process 0 records I0311 08:23:15.390002 1 tasks_processing.go:74] worker 26 stopped. I0311 08:23:15.390008 1 tasks_processing.go:74] worker 10 stopped. I0311 08:23:15.390013 1 tasks_processing.go:74] worker 18 stopped. I0311 08:23:15.390018 1 tasks_processing.go:74] worker 12 stopped. I0311 08:23:15.390121 1 tasks_processing.go:74] worker 25 stopped. I0311 08:23:15.390175 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 17.958102ms to process 0 records I0311 08:23:15.390298 1 tasks_processing.go:74] worker 21 stopped. I0311 08:23:15.390773 1 recorder.go:70] Recording config/oauth with fingerprint=ac5ed7636e87349be5251e4f96d04a8176413d4e0e9456e72f33cd72e3918ade I0311 08:23:15.390794 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 12.837052ms to process 1 records I0311 08:23:15.390980 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0311 08:23:15.391093 1 recorder.go:70] Recording config/authentication with fingerprint=a3bbc63870b53263344721e4e9ec7c83a35f068442ba74d8e8bb2c5c3f92ecf1 I0311 08:23:15.391144 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0311 08:23:15.391152 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 19.315349ms to process 1 records I0311 08:23:15.391129 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0311 08:23:15.391121 1 operator.go:287] started I0311 08:23:15.391160 1 tasks_processing.go:74] worker 8 stopped. I0311 08:23:15.391415 1 tasks_processing.go:74] worker 2 stopped. I0311 08:23:15.391221 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 42.391838ms to process 0 records I0311 08:23:15.391847 1 tasks_processing.go:74] worker 30 stopped. I0311 08:23:15.392566 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=3d9216fa7afa3c0bd96ff9de2316caf6b3058fc9a68aba8ec8433d5e38acf24c I0311 08:23:15.392765 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=3b71305e633b6a24c0f939fa67782490be30c7001670b53ea91deb9393418107 I0311 08:23:15.392775 1 gather.go:177] gatherer "clusterconfig" function "crds" took 28.305491ms to process 2 records I0311 08:23:15.392876 1 tasks_processing.go:74] worker 17 stopped. I0311 08:23:15.393124 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=d5f7846f87911a3655428890d999398285df44baf3116735d30a1b10d411eb97 I0311 08:23:15.393150 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 20.502982ms to process 1 records I0311 08:23:15.393734 1 tasks_processing.go:74] worker 22 stopped. I0311 08:23:15.393752 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 30.266426ms to process 0 records I0311 08:23:15.400299 1 tasks_processing.go:74] worker 4 stopped. I0311 08:23:15.400433 1 recorder.go:70] Recording config/featuregate with fingerprint=8d083a78a3a1b8f611832b15090468390b5e04620d5c636772bc6cc1741eef6c I0311 08:23:15.400445 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 22.366042ms to process 1 records I0311 08:23:15.407188 1 prometheus_rules.go:88] Prometheus rules successfully created I0311 08:23:15.407283 1 controller.go:203] Source scaController *sca.Controller is not ready I0311 08:23:15.407294 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0311 08:23:15.407297 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0311 08:23:15.407301 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0311 08:23:15.407305 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0311 08:23:15.407333 1 controller.go:458] The operator is still being initialized I0311 08:23:15.407341 1 controller.go:481] The operator is healthy I0311 08:23:15.408925 1 tasks_processing.go:74] worker 28 stopped. I0311 08:23:15.409350 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=0a741d9914a51efb4ad3ff2f797020b1b0612f61ebe2f3464f8b870f0e07d760 I0311 08:23:15.409516 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=d1161c0cc45ff166f6530c6a883b3770f87afbf1390f7d46e8a6cd1ea386e85d I0311 08:23:15.409552 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 45.558839ms to process 2 records I0311 08:23:15.409651 1 recorder.go:70] Recording config/olm_operators with fingerprint=10c84bc8dfac3d7ffd30b2f58edcfeeca69588237de9c819d8d7bedd8014b4f4 I0311 08:23:15.409694 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 44.510123ms to process 1 records I0311 08:23:15.409653 1 tasks_processing.go:74] worker 15 stopped. I0311 08:23:15.409890 1 tasks_processing.go:74] worker 7 stopped. I0311 08:23:15.410517 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=d75454218f47d0cdcffeb7a1a303e303ba35d6d250654d7b03075564396ec04b I0311 08:23:15.410868 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=93fe85cff481e888fc04c13181225a362e9a7aff7d887812569922de42a10425 I0311 08:23:15.410943 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=109988f9e23b17ced1d82bf9822c2e16e13d9aa9f8c94cfb7654150149ff1306 I0311 08:23:15.411013 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=83a769dd266b8d4e00ce9f147cdb38c90d1cdd334f4680246c4aae6d766d25be I0311 08:23:15.411123 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=3d534307b00f6376a1956759e984d7ae650f2f1d6c1a01f69a99e65c058269ac I0311 08:23:15.411183 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=180da52f74b909e17e1c85fd23016526775f81c82f5be72b1e123734c3fa0fd7 I0311 08:23:15.411304 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=59daed05e8b441d9982dc7795d17d10783c62d5b305412b9299e50ba956b2ab4 I0311 08:23:15.411363 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=e89cf7bd4b1557d619a23b2473378557bb372a98950ee636a0c264ddd94deb30 I0311 08:23:15.411447 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=ad723be122c2ca5554cf2eb280342e5c0a529f72dacb1fedae097416be78fbc8 I0311 08:23:15.411555 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=57892c7e1a8353cbac5581afe44560f84252838e103e05216116563c33602412 I0311 08:23:15.411586 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 44.58362ms to process 10 records I0311 08:23:15.416910 1 tasks_processing.go:74] worker 6 stopped. E0311 08:23:15.416928 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0311 08:23:15.416936 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0311 08:23:15.416940 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0311 08:23:15.416951 1 recorder.go:70] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=295a2bc7810c628251f1f312c209b5eb4da888326706c635ac0c12aec0969e17 I0311 08:23:15.416981 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0311 08:23:15.416990 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0311 08:23:15.416995 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=fe7abfa7f7aea852d8bca6b7df9b4d5de32045254ff2774f9bd30f0b6dcb7dc4 I0311 08:23:15.417000 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0311 08:23:15.417043 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0311 08:23:15.417054 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0311 08:23:15.417061 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 70.817481ms to process 7 records E0311 08:23:15.419960 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2789008353-f3c0-4c04-b400-9c7056bc2db1%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:48259->172.30.0.10:53: read: connection refused I0311 08:23:15.419977 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2789008353-f3c0-4c04-b400-9c7056bc2db1%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:48259->172.30.0.10:53: read: connection refused I0311 08:23:15.421781 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0311 08:23:15.421788 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0311 08:23:15.421794 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0311 08:23:15.426916 1 base_controller.go:82] Caches are synced for ConfigController I0311 08:23:15.426928 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0311 08:23:15.430347 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 08:23:15.453939 1 tasks_processing.go:74] worker 23 stopped. I0311 08:23:15.453978 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0311 08:23:15.453989 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 88.367938ms to process 1 records I0311 08:23:15.454564 1 tasks_processing.go:74] worker 20 stopped. I0311 08:23:15.455051 1 recorder.go:70] Recording config/version with fingerprint=5c90bf36b839a64f84e34d6fbfeb1cd8c5b91eadf08ad8db18d3e3a5e5964c8d I0311 08:23:15.455064 1 recorder.go:70] Recording config/id with fingerprint=53f079182f39d501443f0ac8add3189a35b6708935366667f0744abbbf5e3503 I0311 08:23:15.455071 1 gather.go:177] gatherer "clusterconfig" function "version" took 104.432775ms to process 2 records I0311 08:23:15.458963 1 tasks_processing.go:74] worker 16 stopped. I0311 08:23:15.459366 1 recorder.go:70] Recording config/node/ip-10-0-0-55.ec2.internal with fingerprint=c2c22efcd664231d87900d6a10a7574f807fd68ae41543010096bd235d32a7b7 I0311 08:23:15.459453 1 recorder.go:70] Recording config/node/ip-10-0-1-58.ec2.internal with fingerprint=a3466e99590bf7b801d4736c5294ca60d1e86abb7dd95f485ccf1e5859b5b161 I0311 08:23:15.459523 1 recorder.go:70] Recording config/node/ip-10-0-2-67.ec2.internal with fingerprint=be4e0bb4cc63518e8c8edacb7f95b20c79b9f65917c1bf59c21acdd1a1af1a0d I0311 08:23:15.459537 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 109.109793ms to process 3 records I0311 08:23:15.469553 1 tasks_processing.go:74] worker 1 stopped. I0311 08:23:15.469572 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 123.453583ms to process 0 records I0311 08:23:15.471360 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0311 08:23:15.474896 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:54185->172.30.0.10:53: read: connection refused I0311 08:23:15.474908 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:54185->172.30.0.10:53: read: connection refused I0311 08:23:15.487519 1 tasks_processing.go:74] worker 9 stopped. E0311 08:23:15.487538 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0311 08:23:15.487544 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ovd18q8bue50eb2boov1bhdddghejth-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ovd18q8bue50eb2boov1bhdddghejth-primary-cert-bundle-secret" not found I0311 08:23:15.487624 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=24d1055b1934b5289f8ac95032bbfcd6187d0d3ef72ebad9a6845e5a43082c58 I0311 08:23:15.487639 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 137.496885ms to process 1 records I0311 08:23:15.491408 1 base_controller.go:82] Caches are synced for LoggingSyncer I0311 08:23:15.491423 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0311 08:23:15.780346 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0311 08:23:15.780363 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0311 08:23:15.780742 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-4hgz8 pod in namespace openshift-dns (previous: false). I0311 08:23:15.800946 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0311 08:23:16.000482 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0311 08:23:16.011539 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-4hgz8 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-4hgz8\" is waiting to start: ContainerCreating" I0311 08:23:16.011554 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-4hgz8\" is waiting to start: ContainerCreating" I0311 08:23:16.011562 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-4hgz8 pod in namespace openshift-dns (previous: false). I0311 08:23:16.186479 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-4hgz8 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-4hgz8\" is waiting to start: ContainerCreating" I0311 08:23:16.186497 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-4hgz8\" is waiting to start: ContainerCreating" I0311 08:23:16.186529 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-j4kbw pod in namespace openshift-dns (previous: false). W0311 08:23:16.382080 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 08:23:16.425750 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-j4kbw pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-j4kbw\" is waiting to start: ContainerCreating" I0311 08:23:16.425768 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-j4kbw\" is waiting to start: ContainerCreating" I0311 08:23:16.425777 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-j4kbw pod in namespace openshift-dns (previous: false). I0311 08:23:16.587632 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-j4kbw pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-j4kbw\" is waiting to start: ContainerCreating" I0311 08:23:16.587653 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-j4kbw\" is waiting to start: ContainerCreating" I0311 08:23:16.587685 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-wn2d9 pod in namespace openshift-dns (previous: false). I0311 08:23:16.804580 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0311 08:23:16.809887 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wn2d9 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-wn2d9\" is waiting to start: ContainerCreating" I0311 08:23:16.809906 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-wn2d9\" is waiting to start: ContainerCreating" I0311 08:23:16.809917 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-wn2d9 pod in namespace openshift-dns (previous: false). I0311 08:23:16.985135 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wn2d9 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-wn2d9\" is waiting to start: ContainerCreating" I0311 08:23:16.985155 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-wn2d9\" is waiting to start: ContainerCreating" I0311 08:23:16.985166 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-9sm62 pod in namespace openshift-dns (previous: false). I0311 08:23:17.184847 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 08:23:17.184869 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-dvfmf pod in namespace openshift-dns (previous: false). I0311 08:23:17.204035 1 tasks_processing.go:74] worker 13 stopped. I0311 08:23:17.204082 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=ade6bd09f8c2837fbe0f78785c00f4683154048f732bb9f41bd455fd5c9d093c I0311 08:23:17.204112 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=8971df6b03f049451b5493da522d0a00777b76aafb822ccdf2114d9031f566be I0311 08:23:17.204149 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0311 08:23:17.204182 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=5bda612660d8616d28bb2497ffc8107eadbf33d361d6478c8a55d6aa4d378f23 I0311 08:23:17.204201 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0311 08:23:17.204249 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=54ce0fcbd07255b2317d1cc0cf4ec99ea2acc5174d3890e7ff768f90eb24230e I0311 08:23:17.204282 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=27b373f9a2323543eba909be5a506f2281e332d94ae8c658e10c9c57810e6985 I0311 08:23:17.204309 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=f287037904304d2348bd371e51884e179c2b3ac2649585492fd431b1693b18bb I0311 08:23:17.204326 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=122fd86f126ecf5267bfc08c343ac7d75ac9e4b0ee165265767a96b7b9f6455e I0311 08:23:17.204345 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=d8cc3d8925010cba4f68555a4383742e9ba0da7705d75e0add8ffdaae1dbd6be I0311 08:23:17.204357 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0311 08:23:17.204374 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=87230c60e2b1be43803ae2a7b0e0440b5022bfa978b2132d11a8c462a67b81fe I0311 08:23:17.204385 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0311 08:23:17.204401 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=5e26899f93e8f261876e44d8af4829d54bdaed1472b31f62476997966e8e1b4f I0311 08:23:17.204411 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0311 08:23:17.204426 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=c84bc7585588497a205852637aaa3a96673443e187c2151924ffb6e854a3de34 I0311 08:23:17.204436 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0311 08:23:17.204452 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=63cc8aa1460cdc833930ef8dcb560e93407105fbae28dbf0040747d0cc7e2cfc I0311 08:23:17.204584 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=4dcdad5af55674a10e997cbb7ef45e762fd7c76dd3b41e5b77e9b69491d3faf7 I0311 08:23:17.204610 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0311 08:23:17.204633 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=835d8f2cd7a50f4998cb49a2ea2eb2af98a0c7d34a50bbefc8b691703e3c38be I0311 08:23:17.204658 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=f7b794034a99208ce59957398c7f64cb566eaea04c40560785fc8a8527a6e3b8 I0311 08:23:17.204669 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0311 08:23:17.204685 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=fef28616778ad83af193e9abab2c3c8f28c992a7a9e7ce87c5f46234cfa434fe I0311 08:23:17.204695 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0311 08:23:17.204709 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=3330462e79cf86605c20690c6020df4770ac2e6e25b5c018b38868c8a43e3592 I0311 08:23:17.204725 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=f628530200b5ee4b29c839a8cf134a513927ace7cdf05726637622c5b956fe43 I0311 08:23:17.204741 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=e227e46c33f6df1220c24120c90854366c3252c1d29c687ae9b61fbbd30d91ce I0311 08:23:17.204757 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=d80a405f0c1e3904cb2259369346b8a14823c9f60a9c7b5a8472a8863c4f68a3 I0311 08:23:17.204771 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=830d08d447946d7dea58370c5e1814824ed4d8c15b26c09364ce5e0039735c75 I0311 08:23:17.204798 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=0c4514e8fee6cda7dba3b958e0f2cb53905a85ae347f57382c2aae1cb95b91ff I0311 08:23:17.204815 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0311 08:23:17.204825 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0311 08:23:17.204833 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.854873881s to process 33 records W0311 08:23:17.383215 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 08:23:17.386272 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 08:23:17.386290 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-htg5k pod in namespace openshift-dns (previous: false). I0311 08:23:17.586025 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 08:23:17.586117 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6d5c5b65db-dnk8r pod in namespace openshift-image-registry (previous: false). I0311 08:23:17.786504 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6d5c5b65db-dnk8r pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6d5c5b65db-dnk8r\" is waiting to start: ContainerCreating" I0311 08:23:17.786523 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6d5c5b65db-dnk8r\" is waiting to start: ContainerCreating" I0311 08:23:17.786580 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-77ffd6644f-d6dfl pod in namespace openshift-image-registry (previous: false). I0311 08:23:17.984960 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-77ffd6644f-d6dfl pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-77ffd6644f-d6dfl\" is waiting to start: ContainerCreating" I0311 08:23:17.984981 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-77ffd6644f-d6dfl\" is waiting to start: ContainerCreating" I0311 08:23:17.985031 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-77ffd6644f-mnr8p pod in namespace openshift-image-registry (previous: false). I0311 08:23:18.186157 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-77ffd6644f-mnr8p pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-77ffd6644f-mnr8p\" is waiting to start: ContainerCreating" I0311 08:23:18.186182 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-77ffd6644f-mnr8p\" is waiting to start: ContainerCreating" I0311 08:23:18.186197 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-6hc4v pod in namespace openshift-image-registry (previous: false). W0311 08:23:18.382140 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 08:23:18.384568 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 08:23:18.384586 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-8ndtf pod in namespace openshift-image-registry (previous: false). I0311 08:23:18.585736 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 08:23:18.585754 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-g2ccd pod in namespace openshift-image-registry (previous: false). I0311 08:23:18.787804 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 08:23:18.787826 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-549f6b67c5-6h47l pod in namespace openshift-ingress (previous: false). I0311 08:23:18.986346 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-549f6b67c5-6h47l pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-549f6b67c5-6h47l\" is waiting to start: ContainerCreating" I0311 08:23:18.986365 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-549f6b67c5-6h47l\" is waiting to start: ContainerCreating" I0311 08:23:18.986377 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-769cbd8f7-dpmkp pod in namespace openshift-ingress (previous: false). I0311 08:23:19.185190 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-769cbd8f7-dpmkp pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-769cbd8f7-dpmkp\" is waiting to start: ContainerCreating" I0311 08:23:19.185210 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-769cbd8f7-dpmkp\" is waiting to start: ContainerCreating" I0311 08:23:19.185220 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-769cbd8f7-jx7vg pod in namespace openshift-ingress (previous: false). W0311 08:23:19.382353 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 08:23:19.386491 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-769cbd8f7-jx7vg pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-769cbd8f7-jx7vg\" is waiting to start: ContainerCreating" I0311 08:23:19.386507 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-769cbd8f7-jx7vg\" is waiting to start: ContainerCreating" I0311 08:23:19.386534 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-55cvn pod in namespace openshift-ingress-canary (previous: false). I0311 08:23:19.585423 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-55cvn pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-55cvn\" is waiting to start: ContainerCreating" I0311 08:23:19.585444 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-55cvn\" is waiting to start: ContainerCreating" I0311 08:23:19.585474 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-6wn9s pod in namespace openshift-ingress-canary (previous: false). I0311 08:23:19.787140 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-6wn9s pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-6wn9s\" is waiting to start: ContainerCreating" I0311 08:23:19.787176 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-6wn9s\" is waiting to start: ContainerCreating" I0311 08:23:19.787204 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-clxmv pod in namespace openshift-ingress-canary (previous: false). I0311 08:23:19.985985 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-clxmv pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-clxmv\" is waiting to start: ContainerCreating" I0311 08:23:19.986009 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-clxmv\" is waiting to start: ContainerCreating" I0311 08:23:19.986041 1 tasks_processing.go:74] worker 5 stopped. I0311 08:23:19.986126 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=f64aac4e68fe577fa2ef52d272d81f48a2007c31370cda743d855ea0955aab8a I0311 08:23:19.986179 1 recorder.go:70] Recording events/openshift-dns with fingerprint=9ce54190b71268eb09fec81a918219f3612f855ebc542ba5be69daaf49a6dd08 I0311 08:23:19.986282 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=cf879780948e35fe7da46023c370be05b7d632bd924c37519fec646dbde23aee I0311 08:23:19.986311 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=fb8c5e836d3b656a5e5d103f2b8bcb0a27a1b7bfb45dec1591e13220c8f33f6f I0311 08:23:19.986367 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=6bfcdf99136016f4fdf8f5b71492f423bb3506172f19684dd3c3889ddbb40e6c I0311 08:23:19.986385 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=7361607ecaa23bfd8b048f0674150910af232192ee68ee0fcce5416822986991 I0311 08:23:19.987918 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-4hgz8 with fingerprint=3211a938e1a2ed72f19d2843fa45dc1af7bc67134cc349a291175dd4bb3f9d71 I0311 08:23:19.988008 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-j4kbw with fingerprint=40f79738ce8c84796880d99bae90d3cbf61dacca5080dcdef194e95789dd443f I0311 08:23:19.988078 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-wn2d9 with fingerprint=952d9d962ade47191c99f66268da7ef463b9cc4b60bf3a1de7cf5fa3b43b675b I0311 08:23:19.988182 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-6d5c5b65db-dnk8r with fingerprint=5c12ca914161f4b3a1609467b6e63ef5f2030ace5be7155387e59dcf8b1d7076 I0311 08:23:19.988289 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-77ffd6644f-d6dfl with fingerprint=503498f2d9e4492a15f90b0bf64f2964a30ac8a84ef5e05818f5eb3750d6f190 I0311 08:23:19.988378 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-77ffd6644f-mnr8p with fingerprint=b186ca8cf850e737828fb757ecaf41203af5cbaf0bca838353521ef5dd7bd9d1 I0311 08:23:19.988436 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-55cvn with fingerprint=f5fe1b7ddac6ce00b934df9ce5dc9f2616bc9507cc234fef6ef3077722b24d1b I0311 08:23:19.988507 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-6wn9s with fingerprint=4b4834509019592a6255f1c7aceb1bc69db47585cec72fa912cfb5f68cf988bb I0311 08:23:19.988564 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-clxmv with fingerprint=f5cf5a4f2e19ae07dc4c4c9b13e2458b6eda67af9a132a1169344b8d0c45f9d0 I0311 08:23:19.988574 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.636658058s to process 15 records W0311 08:23:20.381693 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0311 08:23:20.381718 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0311 08:23:20.381733 1 tasks_processing.go:74] worker 27 stopped. E0311 08:23:20.381745 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0311 08:23:20.381757 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0311 08:23:20.381771 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0311 08:23:20.381783 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.030002944s to process 1 records I0311 08:23:27.858726 1 tasks_processing.go:74] worker 24 stopped. I0311 08:23:27.858775 1 recorder.go:70] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0311 08:23:27.858789 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.494029024s to process 1 records I0311 08:23:28.133943 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 08:23:28.552083 1 tasks_processing.go:74] worker 31 stopped. I0311 08:23:28.552377 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=6a25405fe5ae5697551c08cd61c8ee577eba432556fcc5dc2605b343adedffa8 I0311 08:23:28.552394 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.20615421s to process 1 records E0311 08:23:28.552467 1 periodic.go:250] "Unhandled Error" err="clusterconfig failed after 13.206s with: function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0311 08:23:28.553577 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0311 08:23:28.553595 1 periodic.go:212] Running workloads gatherer I0311 08:23:28.553613 1 tasks_processing.go:45] number of workers: 2 I0311 08:23:28.553628 1 tasks_processing.go:69] worker 1 listening for tasks. I0311 08:23:28.553635 1 tasks_processing.go:71] worker 1 working on workload_info task. I0311 08:23:28.553641 1 tasks_processing.go:69] worker 0 listening for tasks. I0311 08:23:28.553713 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0311 08:23:28.574501 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0311 08:23:28.578350 1 tasks_processing.go:74] worker 0 stopped. I0311 08:23:28.578366 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 24.61842ms to process 0 records I0311 08:23:28.583552 1 gather_workloads_info.go:387] No image sha256:a7c71f3c9714cf1717c45e12ec817be6d6209f00989e6a31d634e527f0c8147d (10ms) I0311 08:23:28.593250 1 gather_workloads_info.go:387] No image sha256:bae7a33f8db8a3d4b3c4c05498aba85a0ce463f85322067c48e79663710e616e (10ms) I0311 08:23:28.604765 1 gather_workloads_info.go:387] No image sha256:ceab8a8c340801fcebf800620d3fef5493d8a443077a21c6e56984051bd3abde (11ms) I0311 08:23:28.618103 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (13ms) I0311 08:23:28.629030 1 gather_workloads_info.go:387] No image sha256:8ef4bdf07dda423fab73484dbc66d527ce8ba8cc2d4f99210bc1bc24ea08c0cb (11ms) I0311 08:23:28.637649 1 gather_workloads_info.go:387] No image sha256:0a92dd43975f972e3dc707fb37854e046813531553038147e9c90d54a8d9df73 (9ms) I0311 08:23:28.647688 1 gather_workloads_info.go:387] No image sha256:2d077e73dd76873ee2a1583aa3b3e76a1b408737f0af2e22b2aa055604d89e81 (10ms) I0311 08:23:28.656728 1 gather_workloads_info.go:387] No image sha256:f2c46054a8f64a0e949cbf30295f31b4a35a0203c6ad03fa7ec922b4101dcbf8 (9ms) I0311 08:23:28.668084 1 gather_workloads_info.go:387] No image sha256:8b1e32ce43eb1247b0558230b6e0baa85ac62f1a5e2a2089d40b4fea90538529 (11ms) I0311 08:23:28.683806 1 gather_workloads_info.go:387] No image sha256:23c4fe84047a4e6cbe7f75f470e7d6fe0e61e7910dd17945b1b61bc4b72f3f2a (16ms) I0311 08:23:28.693500 1 gather_workloads_info.go:387] No image sha256:18956d5fbc8d0369a26cb1e00eb5c3aff1973ae8cbd3983920a0cf2851f05358 (10ms) I0311 08:23:28.783252 1 gather_workloads_info.go:387] No image sha256:a9e885d6f0456a2cc10f9e5da71fe9403f5e4c639b9b7ea15bd403b272ccb824 (90ms) I0311 08:23:28.883621 1 gather_workloads_info.go:387] No image sha256:ccdda82b4adfa8ee4a84b3bad19693be6a42a9881a7b83788d835912129c49a4 (100ms) I0311 08:23:28.983883 1 gather_workloads_info.go:387] No image sha256:d28734effaadd66434e77a1a5fbe2e8a4ca2066cd9f8868c22ade9475539bfd7 (100ms) I0311 08:23:29.084188 1 gather_workloads_info.go:387] No image sha256:752be69b2262be713df12c47f4bac8c2dafed272c401e9a89f8060f053d68054 (100ms) I0311 08:23:29.183748 1 gather_workloads_info.go:387] No image sha256:373614619b9420b110d1508c5f17e066ce69c4c226fd04d02fbb959d9ba41eb6 (100ms) I0311 08:23:29.284026 1 gather_workloads_info.go:387] No image sha256:2fac754deaeade3456361eed52e344318ff16d04819384432759f0ea35530114 (100ms) I0311 08:23:29.384430 1 gather_workloads_info.go:387] No image sha256:6e67c980b7300e769fb4b2adaaf006d0f8274e43b10586701b204ea5153f15fc (100ms) I0311 08:23:29.483711 1 gather_workloads_info.go:387] No image sha256:3d0634fe58641d5242649c44ddba70ed67fdd6d2dcb5c2261df5cee8b33de9fd (99ms) I0311 08:23:29.583506 1 gather_workloads_info.go:387] No image sha256:824a3b8f78e19aa21a9f6444aefe8e0b624886cca18fe828b13acbab55e6e868 (100ms) I0311 08:23:29.683756 1 gather_workloads_info.go:387] No image sha256:87efe06afd1f04426fca7f86c0f74c4ee75c311ba199ddabd5b849b877bc59fa (100ms) I0311 08:23:29.784561 1 gather_workloads_info.go:387] No image sha256:48404bd61d05dc738ccce4d22e36b30dbcdc6015b06b4afb604fd1baeee35bf2 (101ms) I0311 08:23:29.811897 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 08:23:29.885697 1 gather_workloads_info.go:387] No image sha256:420c12f61c53e54eab2d99476759c0de339fca98ce0a3a782bc6545cc0e97a9c (101ms) I0311 08:23:29.983774 1 gather_workloads_info.go:387] No image sha256:0a8a0473029ace3adbb66f490dbe560f0e7782a38cdf32c1f7dd3e092e1d191e (98ms) I0311 08:23:30.010221 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 08:23:30.083802 1 gather_workloads_info.go:387] No image sha256:944d9261ba7a143131fe8267c172defc5f37acc1cea3d4d373ec6fc5d8bfcc31 (100ms) I0311 08:23:30.183753 1 gather_workloads_info.go:387] No image sha256:76f820bd9bd138d29305d545d7d49bfe63e2923f1fb1ec2e8eac81a388359024 (100ms) I0311 08:23:30.284982 1 gather_workloads_info.go:387] No image sha256:14703f73bb1ca69ea03b726e340b3b68f8e294e1f22c80d1a16666ea3d4a88a3 (101ms) I0311 08:23:30.384974 1 gather_workloads_info.go:387] No image sha256:4f175ee49f51dc4379e9993fecd1657c7a9c4c3dc096772cb198d0212b9eea47 (100ms) I0311 08:23:30.484343 1 gather_workloads_info.go:387] No image sha256:7941b5d9a758b8667e99cd9236f7f96ec036af61df5cbc96e16077d36700d7c7 (99ms) I0311 08:23:30.584586 1 gather_workloads_info.go:387] No image sha256:6d6ebff54b8adac74f4d1b12ac8aa0f16cd7b28370e0d6aa847d1c457e03a5b6 (100ms) I0311 08:23:30.686108 1 gather_workloads_info.go:387] No image sha256:9697bc2258bdfa9ae8c1866cc7eb0b3b46851998db827b121bdf77417a881eb3 (102ms) I0311 08:23:30.686143 1 tasks_processing.go:74] worker 1 stopped. E0311 08:23:30.686155 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0311 08:23:30.686433 1 recorder.go:70] Recording config/workload_info with fingerprint=d721eacd8ee8ed2cae095b031cb535890c3bbeae1bc90381bbe02181393a2b13 I0311 08:23:30.686449 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.132499247s to process 1 records E0311 08:23:30.686483 1 periodic.go:250] "Unhandled Error" err="workloads failed after 2.132s with: function \"workload_info\" failed with an error" I0311 08:23:30.688191 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0311 08:23:30.688205 1 periodic.go:212] Running conditional gatherer I0311 08:23:30.696490 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules I0311 08:23:30.703388 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.9:46609->172.30.0.10:53: read: connection refused E0311 08:23:30.703663 1 conditional_gatherer.go:320] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 08:23:30.703722 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0311 08:23:30.710665 1 conditional_gatherer.go:390] cluster version is '4.19.9' E0311 08:23:30.710678 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710683 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710687 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710690 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710693 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710696 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710699 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710702 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 08:23:30.710705 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0311 08:23:30.710719 1 tasks_processing.go:45] number of workers: 3 I0311 08:23:30.710730 1 tasks_processing.go:69] worker 2 listening for tasks. I0311 08:23:30.710733 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0311 08:23:30.710743 1 tasks_processing.go:69] worker 0 listening for tasks. I0311 08:23:30.710749 1 tasks_processing.go:69] worker 1 listening for tasks. I0311 08:23:30.710757 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0311 08:23:30.710758 1 tasks_processing.go:74] worker 1 stopped. I0311 08:23:30.710768 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0311 08:23:30.710850 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0311 08:23:30.710867 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 820ns to process 1 records I0311 08:23:30.710920 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0311 08:23:30.710938 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.227µs to process 1 records I0311 08:23:30.710955 1 tasks_processing.go:74] worker 0 stopped. I0311 08:23:30.711083 1 tasks_processing.go:74] worker 2 stopped. I0311 08:23:30.711094 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 302.833µs to process 0 records I0311 08:23:30.711113 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.9:46609->172.30.0.10:53: read: connection refused I0311 08:23:30.711128 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0311 08:23:30.731062 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=0f1d025c81481e071d051cb5fe81ef89a0442740cf56cadd2e1b73d01a683195 I0311 08:23:30.731213 1 diskrecorder.go:70] Writing 106 records to /var/lib/insights-operator/insights-2026-03-11-082330.tar.gz I0311 08:23:30.737961 1 diskrecorder.go:51] Wrote 106 records to disk in 6ms I0311 08:23:30.737991 1 periodic.go:281] Gathering cluster info every 2h0m0s I0311 08:23:30.738008 1 periodic.go:282] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0311 08:23:46.583819 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 08:24:29.544491 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="c7e4edab00599ea6658cbd1b18e2c2f18c98d91c52950a5a6221acdb664604ed") W0311 08:24:29.544532 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.key was created I0311 08:24:29.544586 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0311 08:24:29.544612 1 base_controller.go:181] Shutting down ConfigController ... I0311 08:24:29.544641 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="02e656208a899c448077005dca257575744307cebfee676d1ced1dac5bc52649") I0311 08:24:29.544728 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" E0311 08:24:29.544732 1 controller.go:290] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled I0311 08:24:29.544742 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0311 08:24:29.544777 1 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0311 08:24:29.544796 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="60d1d02c6b0cee11b00fc464444d2b83e28bbb64fa25ecd14927645fe14e862a") E0311 08:24:29.544801 1 controller.go:290] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled I0311 08:24:29.544651 1 periodic.go:173] Shutting down I0311 08:24:29.544808 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" I0311 08:24:29.544825 1 secure_serving.go:258] Stopped listening on [::]:8443 I0311 08:24:29.544827 1 requestheader_controller.go:194] Shutting down RequestHeaderAuthRequestController