W0312 16:07:02.983537 1 cmd.go:257] Using insecure, self-signed certificates I0312 16:07:03.258814 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0312 16:07:03.259110 1 observer_polling.go:159] Starting file observer W0312 16:07:03.279364 1 builder.go:272] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-5ff5cb4f99" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0312 16:07:03.646899 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0312 16:07:03.647132 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0312 16:07:03.647675 1 secure_serving.go:57] Forcing use of http/1.1 only W0312 16:07:03.647695 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0312 16:07:03.647701 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0312 16:07:03.647705 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0312 16:07:03.647708 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0312 16:07:03.647710 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0312 16:07:03.647712 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0312 16:07:03.647793 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0312 16:07:03.654157 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes CPMSMachineNamePrefix ChunkSizeMiB CloudDualStackNodeIPs ConsolePluginContentSecurityPolicy DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags GatewayAPI GatewayAPIController HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy OnClusterBuild PersistentIPsForVirtualization PrivateHostedZoneAWS RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController VSphereDriverConfiguration VSphereMultiVCenters ValidatingAdmissionPolicy AWSClusterHostedDNS AutomatedEtcdBackup BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNS GCPCustomAPIEndpoints HighlyAvailableArbiter ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements SELinuxChangePolicy SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerification SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0312 16:07:03.654170 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ChunkSizeMiB", "CloudDualStackNodeIPs", "ConsolePluginContentSecurityPolicy", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "GatewayAPI", "GatewayAPIController", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "OnClusterBuild", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "VSphereDriverConfiguration", "VSphereMultiVCenters", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AutomatedEtcdBackup", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "HighlyAvailableArbiter", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "SELinuxChangePolicy", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} E0312 16:07:03.657426 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.189c23b758deebaa openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdditionalRoutingCapabilities\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"CPMSMachineNamePrefix\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"ConsolePluginContentSecurityPolicy\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"GatewayAPI\", \"GatewayAPIController\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MetricsCollectionProfiles\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"OnClusterBuild\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SetEIPForNLBIngressController\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ClusterVersionOperatorConfiguration\", \"DNSNameResolver\", \"DualReplica\", \"DyanmicServiceEndpointIBMCloud\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"Example2\", \"ExternalOIDCWithUIDAndExtraClaimMappings\", \"GCPClusterHostedDNS\", \"GCPCustomAPIEndpoints\", \"HighlyAvailableArbiter\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"KMSEncryptionProvider\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NewOLMCatalogdAPIV1Metas\", \"NewOLMOwnSingleNamespace\", \"NewOLMPreflightPermissionChecks\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"SELinuxChangePolicy\", \"SELinuxMount\", \"ShortCertRotation\", \"SignatureStores\", \"SigstoreImageVerification\", \"SigstoreImageVerificationPKI\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereConfigurableMaxAllowedBlockVolumesPerNode\", \"VSphereHostVMGroupZonal\", \"VSphereMultiDisk\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-03-12 16:07:03.654124458 +0000 UTC m=+0.714752756,LastTimestamp:2026-03-12 16:07:03.654124458 +0000 UTC m=+0.714752756,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" I0312 16:07:03.660839 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0312 16:07:03.660852 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0312 16:07:03.660884 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0312 16:07:03.660891 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0312 16:07:03.660891 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0312 16:07:03.660905 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0312 16:07:03.661096 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-2898824493/tls.crt::/tmp/serving-cert-2898824493/tls.key" I0312 16:07:03.661336 1 secure_serving.go:213] Serving securely on [::]:8443 I0312 16:07:03.661373 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0312 16:07:03.663240 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0312 16:07:03.663271 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0312 16:07:03.663379 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0312 16:07:03.675303 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0312 16:07:03.675329 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0312 16:07:03.680102 1 secretconfigobserver.go:119] support secret does not exist I0312 16:07:03.690146 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0312 16:07:03.702937 1 secretconfigobserver.go:119] support secret does not exist I0312 16:07:03.704735 1 recorder.go:156] Pruning old reports every 7h18m50s, max age is 288h0m0s I0312 16:07:03.717963 1 periodic.go:212] Running clusterconfig gatherer I0312 16:07:03.717973 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0312 16:07:03.717991 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0312 16:07:03.718010 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0312 16:07:03.718021 1 insightsreport.go:296] Starting report retriever I0312 16:07:03.718028 1 tasks_processing.go:45] number of workers: 32 I0312 16:07:03.718030 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0312 16:07:03.718046 1 tasks_processing.go:69] worker 0 listening for tasks. I0312 16:07:03.718056 1 tasks_processing.go:69] worker 1 listening for tasks. I0312 16:07:03.718062 1 tasks_processing.go:69] worker 12 listening for tasks. I0312 16:07:03.718065 1 tasks_processing.go:69] worker 8 listening for tasks. I0312 16:07:03.718071 1 tasks_processing.go:69] worker 9 listening for tasks. I0312 16:07:03.718072 1 tasks_processing.go:71] worker 12 working on node_logs task. I0312 16:07:03.718080 1 tasks_processing.go:71] worker 9 working on monitoring_persistent_volumes task. I0312 16:07:03.718079 1 tasks_processing.go:69] worker 2 listening for tasks. I0312 16:07:03.718082 1 tasks_processing.go:69] worker 5 listening for tasks. I0312 16:07:03.718072 1 tasks_processing.go:71] worker 8 working on openshift_machine_api_events task. I0312 16:07:03.718091 1 tasks_processing.go:69] worker 6 listening for tasks. I0312 16:07:03.718100 1 tasks_processing.go:69] worker 7 listening for tasks. I0312 16:07:03.718103 1 tasks_processing.go:69] worker 3 listening for tasks. I0312 16:07:03.718112 1 tasks_processing.go:69] worker 23 listening for tasks. I0312 16:07:03.718122 1 tasks_processing.go:69] worker 18 listening for tasks. I0312 16:07:03.718122 1 tasks_processing.go:69] worker 17 listening for tasks. I0312 16:07:03.718130 1 tasks_processing.go:69] worker 19 listening for tasks. I0312 16:07:03.718131 1 tasks_processing.go:69] worker 31 listening for tasks. I0312 16:07:03.718138 1 tasks_processing.go:69] worker 20 listening for tasks. I0312 16:07:03.718148 1 tasks_processing.go:69] worker 21 listening for tasks. I0312 16:07:03.718152 1 tasks_processing.go:69] worker 29 listening for tasks. I0312 16:07:03.718134 1 tasks_processing.go:69] worker 13 listening for tasks. I0312 16:07:03.718162 1 tasks_processing.go:69] worker 14 listening for tasks. I0312 16:07:03.718157 1 tasks_processing.go:69] worker 22 listening for tasks. I0312 16:07:03.718168 1 tasks_processing.go:69] worker 30 listening for tasks. I0312 16:07:03.718175 1 tasks_processing.go:69] worker 25 listening for tasks. I0312 16:07:03.718176 1 tasks_processing.go:69] worker 24 listening for tasks. I0312 16:07:03.718175 1 tasks_processing.go:69] worker 15 listening for tasks. I0312 16:07:03.718182 1 tasks_processing.go:69] worker 16 listening for tasks. I0312 16:07:03.718184 1 tasks_processing.go:71] worker 0 working on overlapping_namespace_uids task. I0312 16:07:03.718188 1 tasks_processing.go:69] worker 11 listening for tasks. I0312 16:07:03.718191 1 tasks_processing.go:71] worker 1 working on certificate_signing_requests task. I0312 16:07:03.718198 1 tasks_processing.go:69] worker 10 listening for tasks. I0312 16:07:03.718227 1 tasks_processing.go:69] worker 26 listening for tasks. I0312 16:07:03.718237 1 tasks_processing.go:69] worker 27 listening for tasks. I0312 16:07:03.718072 1 tasks_processing.go:69] worker 4 listening for tasks. I0312 16:07:03.718334 1 tasks_processing.go:71] worker 4 working on ingress task. I0312 16:07:03.718619 1 tasks_processing.go:71] worker 2 working on nodes task. I0312 16:07:03.718656 1 tasks_processing.go:71] worker 7 working on lokistack task. I0312 16:07:03.718698 1 tasks_processing.go:71] worker 3 working on ingress_certificates task. I0312 16:07:03.718623 1 tasks_processing.go:69] worker 28 listening for tasks. I0312 16:07:03.718631 1 tasks_processing.go:71] worker 29 working on image_registries task. I0312 16:07:03.719001 1 tasks_processing.go:71] worker 28 working on pdbs task. I0312 16:07:03.718637 1 tasks_processing.go:71] worker 17 working on version task. I0312 16:07:03.718638 1 tasks_processing.go:71] worker 18 working on openstack_controlplanes task. I0312 16:07:03.718642 1 tasks_processing.go:71] worker 19 working on ceph_cluster task. I0312 16:07:03.718645 1 tasks_processing.go:71] worker 5 working on metrics task. I0312 16:07:03.718646 1 tasks_processing.go:71] worker 31 working on aggregated_monitoring_cr_names task. W0312 16:07:03.719479 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0312 16:07:03.719490 1 tasks_processing.go:71] worker 5 working on support_secret task. I0312 16:07:03.719497 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.271µs to process 0 records I0312 16:07:03.718650 1 tasks_processing.go:71] worker 6 working on storage_cluster task. I0312 16:07:03.718650 1 tasks_processing.go:71] worker 20 working on machines task. I0312 16:07:03.718660 1 tasks_processing.go:71] worker 24 working on cost_management_metrics_configs task. I0312 16:07:03.718655 1 tasks_processing.go:71] worker 21 working on cluster_apiserver task. I0312 16:07:03.718664 1 tasks_processing.go:71] worker 13 working on container_images task. I0312 16:07:03.718666 1 tasks_processing.go:71] worker 22 working on openshift_logging task. I0312 16:07:03.718667 1 tasks_processing.go:71] worker 30 working on silenced_alerts task. W0312 16:07:03.719946 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0312 16:07:03.718671 1 tasks_processing.go:71] worker 25 working on nodenetworkconfigurationpolicies task. I0312 16:07:03.719967 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 27.269µs to process 0 records I0312 16:07:03.718671 1 tasks_processing.go:71] worker 14 working on operators_pods_and_events task. I0312 16:07:03.718681 1 tasks_processing.go:71] worker 26 working on authentication task. I0312 16:07:03.718680 1 tasks_processing.go:71] worker 11 working on config_maps task. I0312 16:07:03.718685 1 tasks_processing.go:71] worker 15 working on service_accounts task. I0312 16:07:03.718684 1 tasks_processing.go:71] worker 10 working on infrastructures task. I0312 16:07:03.718688 1 tasks_processing.go:71] worker 16 working on tsdb_status task. W0312 16:07:03.720440 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0312 16:07:03.720454 1 tasks_processing.go:71] worker 16 working on networks task. I0312 16:07:03.718694 1 tasks_processing.go:71] worker 27 working on image_pruners task. I0312 16:07:03.718705 1 tasks_processing.go:71] worker 23 working on oauths task. I0312 16:07:03.720623 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 30.595µs to process 0 records I0312 16:07:03.719957 1 tasks_processing.go:71] worker 30 working on machine_config_pools task. I0312 16:07:03.722384 1 tasks_processing.go:71] worker 7 working on operators task. I0312 16:07:03.722397 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 3.671818ms to process 0 records I0312 16:07:03.724195 1 tasks_processing.go:71] worker 18 working on machine_configs task. I0312 16:07:03.724239 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 4.862254ms to process 0 records I0312 16:07:03.724805 1 tasks_processing.go:71] worker 24 working on machine_autoscalers task. I0312 16:07:03.724816 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 5.095884ms to process 0 records E0312 16:07:03.724835 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0312 16:07:03.724843 1 gather.go:177] gatherer "clusterconfig" function "machines" took 5.245161ms to process 0 records I0312 16:07:03.724886 1 tasks_processing.go:71] worker 20 working on machine_sets task. I0312 16:07:03.725628 1 tasks_processing.go:71] worker 19 working on jaegers task. I0312 16:07:03.726044 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 6.209348ms to process 0 records I0312 16:07:03.726111 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 6.48479ms to process 0 records I0312 16:07:03.726142 1 tasks_processing.go:71] worker 6 working on pod_network_connectivity_checks task. I0312 16:07:03.727092 1 tasks_processing.go:71] worker 25 working on openstack_dataplanedeployments task. I0312 16:07:03.727115 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 7.123618ms to process 0 records I0312 16:07:03.727868 1 tasks_processing.go:71] worker 22 working on sap_config task. I0312 16:07:03.727881 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 7.96602ms to process 0 records I0312 16:07:03.727894 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 3.056834ms to process 0 records I0312 16:07:03.727901 1 tasks_processing.go:71] worker 24 working on crds task. I0312 16:07:03.732338 1 tasks_processing.go:71] worker 30 working on validating_webhook_configurations task. I0312 16:07:03.732368 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 11.579915ms to process 0 records I0312 16:07:03.732427 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 7.464573ms to process 0 records I0312 16:07:03.732435 1 tasks_processing.go:71] worker 20 working on sap_datahubs task. I0312 16:07:03.735741 1 tasks_processing.go:71] worker 9 working on schedulers task. I0312 16:07:03.735757 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 17.650649ms to process 0 records I0312 16:07:03.735909 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0312 16:07:03.735934 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0312 16:07:03.735942 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0312 16:07:03.735947 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0312 16:07:03.735966 1 controller.go:458] The operator is still being initialized I0312 16:07:03.735974 1 controller.go:481] The operator is healthy I0312 16:07:03.736051 1 tasks_processing.go:71] worker 4 working on nodenetworkstates task. I0312 16:07:03.736427 1 recorder.go:70] Recording config/ingress with fingerprint=ece5c505a7eaefa6e9792ccf829da78109348679a66a3a64c502669bc2ce6e86 I0312 16:07:03.736447 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 17.664415ms to process 1 records I0312 16:07:03.737740 1 tasks_processing.go:71] worker 1 working on dvo_metrics task. I0312 16:07:03.737752 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 19.533012ms to process 0 records I0312 16:07:03.738525 1 tasks_processing.go:71] worker 19 working on proxies task. I0312 16:07:03.738535 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 12.839437ms to process 0 records I0312 16:07:03.743538 1 tasks_processing.go:71] worker 20 working on mutating_webhook_configurations task. I0312 16:07:03.743555 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 11.095526ms to process 0 records I0312 16:07:03.743707 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 15.708467ms to process 0 records I0312 16:07:03.743718 1 tasks_processing.go:71] worker 22 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0312 16:07:03.743751 1 tasks_processing.go:71] worker 6 working on sap_pods task. E0312 16:07:03.743720 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0312 16:07:03.743826 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 17.427639ms to process 0 records I0312 16:07:03.743861 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 16.509915ms to process 0 records E0312 16:07:03.743892 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0312 16:07:03.743928 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0312 16:07:03.743934 1 tasks_processing.go:71] worker 25 working on container_runtime_configs task. I0312 16:07:03.743940 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 19.469327ms to process 1 records I0312 16:07:03.743981 1 tasks_processing.go:71] worker 18 working on storage_classes task. I0312 16:07:03.744067 1 tasks_processing.go:71] worker 28 working on machine_healthchecks task. I0312 16:07:03.744080 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=f0c8dabaaf3db055f33f7ef085f2d28dbf28dd9312fe336c4eae527cf8b4b5d8 I0312 16:07:03.744255 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=024ea74c939969829cbfa0d7a96996100d97276c134af8b1e1ce1681b95ff1ad I0312 16:07:03.744285 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=72ac7d530a1ee6e068e6120c8ff5cce35a8cc6f2a93f0ee0ef4c97ed0d4736da I0312 16:07:03.744314 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 24.686501ms to process 3 records I0312 16:07:03.744415 1 tasks_processing.go:71] worker 10 working on install_plans task. I0312 16:07:03.744964 1 recorder.go:70] Recording config/infrastructure with fingerprint=abac98042f3b7b558cd5561aa44d3412535838a444185f4dcd1bb7a54584e586 I0312 16:07:03.744978 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 23.534602ms to process 1 records I0312 16:07:03.760376 1 tasks_processing.go:71] worker 4 working on feature_gates task. I0312 16:07:03.760395 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 24.30731ms to process 0 records I0312 16:07:03.761297 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0312 16:07:03.761316 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0312 16:07:03.761334 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0312 16:07:03.761704 1 tasks_processing.go:71] worker 26 working on image task. I0312 16:07:03.761950 1 recorder.go:70] Recording config/authentication with fingerprint=35d053a56f2f527b0fd6f9bd50e5a1957415898fe3b59be1b18b476d997a3dd8 I0312 16:07:03.761965 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 41.60277ms to process 1 records I0312 16:07:03.763609 1 base_controller.go:82] Caches are synced for ConfigController I0312 16:07:03.763623 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0312 16:07:03.764512 1 tasks_processing.go:71] worker 16 working on olm_operators task. I0312 16:07:03.765341 1 recorder.go:70] Recording config/network with fingerprint=dfac520aa19c025ef02fe7e44c8696b31fc7407df711b2e26288e09e5b34f831 I0312 16:07:03.765414 1 gather.go:177] gatherer "clusterconfig" function "networks" took 44.048754ms to process 1 records I0312 16:07:03.766241 1 tasks_processing.go:71] worker 5 working on openstack_dataplanenodesets task. E0312 16:07:03.766341 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0312 16:07:03.766357 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 46.740366ms to process 0 records E0312 16:07:03.766492 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0312 16:07:03.766541 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 22.381796ms to process 0 records I0312 16:07:03.766501 1 tasks_processing.go:71] worker 28 working on clusterroles task. I0312 16:07:03.766670 1 tasks_processing.go:71] worker 27 working on active_alerts task. W0312 16:07:03.766710 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0312 16:07:03.766989 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=c507923049730c9b6ea2e6db4b308cc32b7b7bf9ac8114922b29a41642309ee1 I0312 16:07:03.767011 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 46.165745ms to process 1 records I0312 16:07:03.767020 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 35.929µs to process 0 records I0312 16:07:03.767028 1 tasks_processing.go:71] worker 27 working on openstack_version task. I0312 16:07:03.767178 1 tasks_processing.go:74] worker 6 stopped. I0312 16:07:03.767191 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 23.415377ms to process 0 records I0312 16:07:03.767453 1 tasks_processing.go:74] worker 2 stopped. I0312 16:07:03.767929 1 recorder.go:70] Recording config/node/ip-10-0-0-16.ec2.internal with fingerprint=760f7b5f6d4bab2add1274893d13fe4ecc545fd21393c9938866ee439f1358b2 I0312 16:07:03.768030 1 recorder.go:70] Recording config/node/ip-10-0-1-71.ec2.internal with fingerprint=7ab21a99d80684f7666890103444c45a5aec197fd268cbb8eed4148a16699fef I0312 16:07:03.768119 1 recorder.go:70] Recording config/node/ip-10-0-2-99.ec2.internal with fingerprint=612b068184cc0b09c8d17ef47dd1569dd76e1a5c0343eb3966b53ccf2c175318 I0312 16:07:03.768136 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 48.78899ms to process 3 records I0312 16:07:03.768331 1 recorder.go:70] Recording config/apiserver with fingerprint=0d5b532b52aec3d16598eb8477baeceac2350316ecf25e5c86a539cfab2d5388 I0312 16:07:03.768348 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 47.971917ms to process 1 records I0312 16:07:03.768358 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 23.822201ms to process 0 records I0312 16:07:03.768430 1 tasks_processing.go:74] worker 21 stopped. I0312 16:07:03.768440 1 tasks_processing.go:74] worker 25 stopped. I0312 16:07:03.768622 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=69cfb82b82f1066b63bf36fbf055aae39d34850da1ed343a5a01359168b624e6 I0312 16:07:03.768810 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=bf6a307faa9035c410011c5b44b8ce351bec6759d14a1af1e388e92a6b169875 I0312 16:07:03.768863 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=100c502d4bba25a0758c45120fa53dff544737021d843517cc81fc74f8c3047b I0312 16:07:03.768932 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=f1824d3f6564525d797adec41ea07ecdc9b6e0ed4e4b7ab8968ff5863595cf56 I0312 16:07:03.769002 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=69023b8dbc12cb296f2548d415d3bd2cc72357d99d2d1306e2b7f663ad132695 I0312 16:07:03.769088 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=ed568b27417f4eb836510dcf2a4b18f0378e2afee352f3714372a42975586681 I0312 16:07:03.769195 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=3f517298c4ae0af48589e19eccfb610a7c4831621699f30bcb1cbfd56dd7ac3c I0312 16:07:03.770783 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=9a2a49eccdfe9a8dbbcbe455fd4d74c9ce583e7ad7e92601cb900ed57f986811 I0312 16:07:03.775665 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=ff6c0b076285e6ce9c4593724db8c07b3cf7a0a9be95c15f1e09c4fdf04d17b3 I0312 16:07:03.769259 1 tasks_processing.go:74] worker 30 stopped. I0312 16:07:03.775793 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=ac09555fc581f70cbd06f9d5ea0b86441075aef4f3de2b5a24e083b46df5447d I0312 16:07:03.775898 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 35.689119ms to process 10 records I0312 16:07:03.775932 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 50.037686ms to process 0 records I0312 16:07:03.775961 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 50.070243ms to process 0 records I0312 16:07:03.776006 1 tasks_processing.go:74] worker 8 stopped. I0312 16:07:03.776023 1 tasks_processing.go:74] worker 12 stopped. I0312 16:07:03.776195 1 tasks_processing.go:74] worker 29 stopped. W0312 16:07:03.776430 1 operator.go:287] started I0312 16:07:03.776564 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0312 16:07:03.776640 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0312 16:07:03.776844 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0312 16:07:03.778349 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=88918417d6cec3e97d7e11c20a4dffb4142f220435ed4627dfdec478a75eaf8e I0312 16:07:03.778379 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 50.245071ms to process 1 records I0312 16:07:03.778404 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0312 16:07:03.778412 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 53.617148ms to process 1 records I0312 16:07:03.778508 1 recorder.go:70] Recording config/proxy with fingerprint=b8ed1f663c8c1c21a4e3a570e4d17a7369eda1081d70f230eed08480a839584a I0312 16:07:03.778526 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 37.462289ms to process 1 records I0312 16:07:03.778533 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 10.162384ms to process 0 records I0312 16:07:03.778553 1 tasks_processing.go:74] worker 5 stopped. I0312 16:07:03.778559 1 tasks_processing.go:74] worker 0 stopped. I0312 16:07:03.778564 1 tasks_processing.go:74] worker 19 stopped. I0312 16:07:03.778809 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 11.736171ms to process 0 records I0312 16:07:03.779671 1 recorder.go:70] Recording config/image with fingerprint=6ba8b8339a1587bf4d4f7c447a259662835b9353bb24ec8c6cd4ff2177a0e40a I0312 16:07:03.778844 1 configmapobserver.go:84] configmaps "insights-config" not found I0312 16:07:03.779703 1 gather.go:177] gatherer "clusterconfig" function "image" took 17.253126ms to process 1 records I0312 16:07:03.778775 1 tasks_processing.go:74] worker 27 stopped. I0312 16:07:03.779749 1 tasks_processing.go:74] worker 26 stopped. I0312 16:07:03.779830 1 tasks_processing.go:74] worker 4 stopped. I0312 16:07:03.779869 1 recorder.go:70] Recording config/featuregate with fingerprint=3ce2b3be993e4f87d7154f98f8d307aafaa53e8fbfd22e9185d9a995a204331c I0312 16:07:03.779927 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 18.589542ms to process 1 records I0312 16:07:03.780031 1 tasks_processing.go:74] worker 18 stopped. I0312 16:07:03.780181 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=9f81d76b768018069ae2a5f987a4617d4a99f682c63822169774a916a16d6dd2 I0312 16:07:03.780271 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=2e9216a8a3a3ba97da8a9c37bf8787f485463c841e274774f9572a708599867f I0312 16:07:03.780309 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 36.013979ms to process 2 records I0312 16:07:03.780441 1 tasks_processing.go:74] worker 23 stopped. I0312 16:07:03.780759 1 recorder.go:70] Recording config/oauth with fingerprint=f2b7d38fc4b4eca476494262f30902df99dcba7691d343b2c2766ab18aca3973 I0312 16:07:03.780780 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 59.851121ms to process 1 records W0312 16:07:03.782734 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0312 16:07:03.783614 1 tasks_processing.go:74] worker 20 stopped. I0312 16:07:03.783723 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=50d60b2e7e01791bf032c534a54dbf9a82368c2c27486749edb3f4b94d9700f9 I0312 16:07:03.783775 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=ac5a8a8903a257152c8e7a006554db6250eb4c8af4c1b865af6be866066c024c I0312 16:07:03.783817 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=5f603746db8a803f8c0065f14151a4b3e6ba32b7bdc8445e74a17b9826344058 I0312 16:07:03.783830 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 40.02378ms to process 3 records I0312 16:07:03.783982 1 tasks_processing.go:74] worker 13 stopped. I0312 16:07:03.785571 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-gwfns with fingerprint=467d715ba09a7a67dba9205c502332d64fa579d871a820b968c14a8b8d584401 I0312 16:07:03.785620 1 recorder.go:70] Recording config/running_containers with fingerprint=d406f650141d611986d5639dfd79fe2b618c50429c828f7cb24251624b2a8a65 I0312 16:07:03.785629 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 64.133722ms to process 2 records I0312 16:07:03.790121 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0312 16:07:03.790140 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0312 16:07:03.790147 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0312 16:07:03.790153 1 controller.go:203] Source scaController *sca.Controller is not ready I0312 16:07:03.790158 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0312 16:07:03.790163 1 tasks_processing.go:74] worker 31 stopped. I0312 16:07:03.790178 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 70.67586ms to process 0 records I0312 16:07:03.790183 1 controller.go:458] The operator is still being initialized I0312 16:07:03.790192 1 controller.go:481] The operator is healthy I0312 16:07:03.790303 1 tasks_processing.go:74] worker 24 stopped. I0312 16:07:03.790838 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=1f4a7d137ae1301fe1763445f07cc4286b1b57353052ba38c6d7cbbcbef7f075 I0312 16:07:03.791090 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=7e2c18a7864a00efd5713aefe93a730f52d7064c4e93fc630d4bbf97cf124d77 I0312 16:07:03.791105 1 gather.go:177] gatherer "clusterconfig" function "crds" took 62.386771ms to process 2 records I0312 16:07:03.796112 1 prometheus_rules.go:88] Prometheus rules successfully created I0312 16:07:03.799264 1 tasks_processing.go:74] worker 28 stopped. I0312 16:07:03.799582 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=2ee0a3971e151d8d8dfebbe83a61f95b78da69199fb2ce3f2badc302b194923a I0312 16:07:03.799752 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=8c500abdea35cb255d810e160d4f4cdaec05f1c4538b9630523987e950f0de13 I0312 16:07:03.799769 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 32.69211ms to process 2 records I0312 16:07:03.799836 1 tasks_processing.go:74] worker 16 stopped. I0312 16:07:03.799925 1 recorder.go:70] Recording config/olm_operators with fingerprint=ed7c007cf3e68e8d6e7b43f796a127441875f34b78904fe098d2fe5a39809979 I0312 16:07:03.799937 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 35.243793ms to process 1 records I0312 16:07:03.802871 1 tasks_processing.go:74] worker 22 stopped. I0312 16:07:03.802888 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 59.135283ms to process 0 records I0312 16:07:03.806567 1 tasks_processing.go:74] worker 17 stopped. I0312 16:07:03.807265 1 recorder.go:70] Recording config/version with fingerprint=cf894756e31f6c8068b1fce68190cb9abef1ffd0fc8877a13b1af011e163f657 I0312 16:07:03.807332 1 recorder.go:70] Recording config/id with fingerprint=e70fc7ad1fd88e5b344dc1c06bc00d1c93d74029d8a58fcadc74d6fb34defa25 I0312 16:07:03.807365 1 gather.go:177] gatherer "clusterconfig" function "version" took 87.315245ms to process 2 records E0312 16:07:03.809844 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%278f8f8919-7f28-431a-90fb-c65df75d00fc%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:37201->172.30.0.10:53: read: connection refused I0312 16:07:03.809859 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%278f8f8919-7f28-431a-90fb-c65df75d00fc%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:37201->172.30.0.10:53: read: connection refused I0312 16:07:03.817313 1 tasks_processing.go:74] worker 11 stopped. E0312 16:07:03.817336 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0312 16:07:03.817397 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0312 16:07:03.817405 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0312 16:07:03.817419 1 recorder.go:70] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=295a2bc7810c628251f1f312c209b5eb4da888326706c635ac0c12aec0969e17 I0312 16:07:03.817460 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0312 16:07:03.817470 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0312 16:07:03.817478 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=fe7abfa7f7aea852d8bca6b7df9b4d5de32045254ff2774f9bd30f0b6dcb7dc4 I0312 16:07:03.817485 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0312 16:07:03.817553 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0312 16:07:03.817567 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0312 16:07:03.817576 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 97.107571ms to process 7 records I0312 16:07:03.857968 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0312 16:07:03.861501 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:37895->172.30.0.10:53: read: connection refused I0312 16:07:03.861519 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:37895->172.30.0.10:53: read: connection refused I0312 16:07:03.876650 1 base_controller.go:82] Caches are synced for LoggingSyncer I0312 16:07:03.876663 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0312 16:07:03.899505 1 tasks_processing.go:74] worker 3 stopped. E0312 16:07:03.899527 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0312 16:07:03.899539 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2p08kqbffbgkvb38d49p3vr3jtpagklm-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2p08kqbffbgkvb38d49p3vr3jtpagklm-primary-cert-bundle-secret" not found I0312 16:07:03.899611 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=a7bdcb0f46d1d1cce1bd3325307507a16bbc9324b0d9bc853910452a64b4e7e6 I0312 16:07:03.899624 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 180.742615ms to process 1 records I0312 16:07:04.203000 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0312 16:07:04.403287 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" W0312 16:07:04.770469 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0312 16:07:04.807812 1 tasks_processing.go:74] worker 9 stopped. I0312 16:07:04.807892 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=9e758efb0a80e682a8101d508ea3f202a84e6801b01fd69bd7b198739ae1b0a8 I0312 16:07:04.807905 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 1.072007361s to process 1 records I0312 16:07:04.976840 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0312 16:07:04.976855 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0312 16:07:04.977325 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-77ng7 pod in namespace openshift-dns (previous: false). I0312 16:07:05.210514 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-77ng7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-77ng7\" is waiting to start: ContainerCreating" I0312 16:07:05.210536 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-77ng7\" is waiting to start: ContainerCreating" I0312 16:07:05.210546 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-77ng7 pod in namespace openshift-dns (previous: false). I0312 16:07:05.222481 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0312 16:07:05.382124 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-77ng7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-77ng7\" is waiting to start: ContainerCreating" I0312 16:07:05.382141 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-77ng7\" is waiting to start: ContainerCreating" I0312 16:07:05.382169 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-slbd7 pod in namespace openshift-dns (previous: false). I0312 16:07:05.603535 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-slbd7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-slbd7\" is waiting to start: ContainerCreating" I0312 16:07:05.603554 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-slbd7\" is waiting to start: ContainerCreating" I0312 16:07:05.603565 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-slbd7 pod in namespace openshift-dns (previous: false). I0312 16:07:05.608629 1 tasks_processing.go:74] worker 7 stopped. I0312 16:07:05.608680 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=201e5c3c1ea3e344737498dd6109991514ebaf223406a835c15d37bd0e267a68 I0312 16:07:05.608717 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=b9000e0a6d315a4a88ba679d6854f45bde6084bb5970244667a191b692ea8035 I0312 16:07:05.608751 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0312 16:07:05.608783 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=c07e59e1d4d6bd77cf5431c656a9f20cbf3a5cf165b46bb70ee127b6e1cc296f I0312 16:07:05.608804 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0312 16:07:05.608829 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=e72a05d99fbf3a02ea2629cf383d69f7e920fe94f01566088d9cec105a981468 I0312 16:07:05.608861 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=a5a87b1d2377226bb640661d3f0936839375cf3f0327707e5ef9e5dfca39eba5 I0312 16:07:05.608887 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=a8196990e7ccc40f8aeab64756b7c8c102c05d80e882aaf6ab878a449bceea75 I0312 16:07:05.608901 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=134d9b75a960ab78ef5958b3b895e1b6d515016afba8110eccc488162ac84ea4 I0312 16:07:05.608921 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=e187f2acd3a77c0f6f3d02c1f68e5fd66b4a376d0940d914631e053a5fda12cb I0312 16:07:05.608931 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0312 16:07:05.608947 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=3cc3da57315764efe46cf588697d0b5d0aa89396e3b7960fa8e8c84ebf478528 I0312 16:07:05.608960 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0312 16:07:05.608977 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=5b2310effb25cfd72bdebb73b2d0dd7b08121e8e219260555de94bf4c389becb I0312 16:07:05.608988 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0312 16:07:05.609009 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=7f17d4faa2b80853720c739ec8e9f6e4290e75a4e2e3e15311975a4bad2e6aa3 I0312 16:07:05.609020 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0312 16:07:05.609036 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=0b38eac8399a568b8a6622f4560f74f8524a2254a0f16bcdc46fe9f3de42b713 I0312 16:07:05.609170 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=579afa308335130b515d7230664090f7869d33e07a4a6e5024a96dcdaa6f960b I0312 16:07:05.609227 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0312 16:07:05.609266 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=379554399876c6625578d04ecabf2cc55abefcf24cfe18cfa0b8d761ed7892f9 I0312 16:07:05.609303 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=abb46a98442682b4dee3286862320169bda6e5f25055601cddf63029a09203ba I0312 16:07:05.609321 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0312 16:07:05.609353 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=18cbf620425f10010b49be6d7f3da07311dc64d918a3ca40013b30f49f10a683 I0312 16:07:05.609368 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0312 16:07:05.609384 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=12a11f7085e856fe9006a8bd2497fdd5806627b0ba16eb5a25ed7a54d8afeabf I0312 16:07:05.609399 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=efd74cf89fb952f1f63e29d33303a36f04c0638009fa0d1d145fea0b7debc829 I0312 16:07:05.609415 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=33fa1e24364368372c70b8224eb77efa25ebb121232208ff14564de4218a3c77 I0312 16:07:05.609431 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=4f1f83d056c224a03a005ee3efdebf70ef071ebbf2c7113077726c469bd5fd75 I0312 16:07:05.609444 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=6e5f62f506a2ea5965330a751778e09a82d72a6d4c36e8de46bbdb28f5d24d3e I0312 16:07:05.609470 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=b4ab9e39a217b42e0fe79e9a7a07db451f76b07db292cb77f9a0f9d0efac1c6a I0312 16:07:05.609487 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0312 16:07:05.609498 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0312 16:07:05.609506 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.886222853s to process 33 records W0312 16:07:05.770410 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0312 16:07:05.786313 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-slbd7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-slbd7\" is waiting to start: ContainerCreating" I0312 16:07:05.786333 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-slbd7\" is waiting to start: ContainerCreating" I0312 16:07:05.786373 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-xtptc pod in namespace openshift-dns (previous: false). I0312 16:07:06.003993 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xtptc pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-xtptc\" is waiting to start: ContainerCreating" I0312 16:07:06.004009 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-xtptc\" is waiting to start: ContainerCreating" I0312 16:07:06.004022 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-xtptc pod in namespace openshift-dns (previous: false). I0312 16:07:06.182152 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xtptc pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-xtptc\" is waiting to start: ContainerCreating" I0312 16:07:06.182170 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-xtptc\" is waiting to start: ContainerCreating" I0312 16:07:06.182183 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-2662h pod in namespace openshift-dns (previous: false). I0312 16:07:06.387844 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0312 16:07:06.387862 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-49498 pod in namespace openshift-dns (previous: false). I0312 16:07:06.581618 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0312 16:07:06.581636 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-d7wlc pod in namespace openshift-dns (previous: false). W0312 16:07:06.770825 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0312 16:07:06.781136 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0312 16:07:06.781245 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6cb9fd6766-p4zcc pod in namespace openshift-image-registry (previous: false). I0312 16:07:06.981771 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6cb9fd6766-p4zcc pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6cb9fd6766-p4zcc\" is waiting to start: ContainerCreating" I0312 16:07:06.981792 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6cb9fd6766-p4zcc\" is waiting to start: ContainerCreating" I0312 16:07:06.981843 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7776cbcf7c-lkjjg pod in namespace openshift-image-registry (previous: false). I0312 16:07:07.181865 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7776cbcf7c-lkjjg pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7776cbcf7c-lkjjg\" is waiting to start: ContainerCreating" I0312 16:07:07.181884 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7776cbcf7c-lkjjg\" is waiting to start: ContainerCreating" I0312 16:07:07.181921 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7776cbcf7c-svgj7 pod in namespace openshift-image-registry (previous: false). I0312 16:07:07.384338 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7776cbcf7c-svgj7 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7776cbcf7c-svgj7\" is waiting to start: ContainerCreating" I0312 16:07:07.384356 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7776cbcf7c-svgj7\" is waiting to start: ContainerCreating" I0312 16:07:07.384370 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9tcfp pod in namespace openshift-image-registry (previous: false). I0312 16:07:07.582436 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0312 16:07:07.582456 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-hpnf2 pod in namespace openshift-image-registry (previous: false). W0312 16:07:07.771843 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0312 16:07:07.784448 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0312 16:07:07.784467 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-pqxcn pod in namespace openshift-image-registry (previous: false). I0312 16:07:07.985042 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0312 16:07:07.985066 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6d96d798dc-5mrqm pod in namespace openshift-ingress (previous: false). I0312 16:07:08.182969 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6d96d798dc-5mrqm pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6d96d798dc-5mrqm\" is waiting to start: ContainerCreating" I0312 16:07:08.182992 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6d96d798dc-5mrqm\" is waiting to start: ContainerCreating" I0312 16:07:08.183006 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6d96d798dc-p2jwd pod in namespace openshift-ingress (previous: false). I0312 16:07:08.382956 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6d96d798dc-p2jwd pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6d96d798dc-p2jwd\" is waiting to start: ContainerCreating" I0312 16:07:08.382974 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6d96d798dc-p2jwd\" is waiting to start: ContainerCreating" I0312 16:07:08.382987 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-b9c69fcd7-8wf6p pod in namespace openshift-ingress (previous: false). I0312 16:07:08.593279 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-b9c69fcd7-8wf6p pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-b9c69fcd7-8wf6p\" is waiting to start: ContainerCreating" I0312 16:07:08.593300 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-b9c69fcd7-8wf6p\" is waiting to start: ContainerCreating" I0312 16:07:08.593331 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-599vg pod in namespace openshift-ingress-canary (previous: false). W0312 16:07:08.771191 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0312 16:07:08.771234 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0312 16:07:08.771250 1 tasks_processing.go:74] worker 1 stopped. E0312 16:07:08.771263 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0312 16:07:08.771273 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0312 16:07:08.771288 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0312 16:07:08.771298 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.033492919s to process 1 records I0312 16:07:08.781359 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-599vg pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-599vg\" is waiting to start: ContainerCreating" I0312 16:07:08.781376 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-599vg\" is waiting to start: ContainerCreating" I0312 16:07:08.781408 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-knlng pod in namespace openshift-ingress-canary (previous: false). I0312 16:07:08.981924 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-knlng pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-knlng\" is waiting to start: ContainerCreating" I0312 16:07:08.981943 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-knlng\" is waiting to start: ContainerCreating" I0312 16:07:08.981976 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-wn647 pod in namespace openshift-ingress-canary (previous: false). I0312 16:07:09.182068 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-wn647 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-wn647\" is waiting to start: ContainerCreating" I0312 16:07:09.182087 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-wn647\" is waiting to start: ContainerCreating" I0312 16:07:09.182101 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-949fdd477-sxvz5 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0312 16:07:09.383447 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-949fdd477-sxvz5 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0312 16:07:09.581388 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-7944b9f44d-smpwz pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0312 16:07:09.784221 1 tasks_processing.go:74] worker 14 stopped. I0312 16:07:09.784343 1 recorder.go:70] Recording events/openshift-dns with fingerprint=3e36a6acf23c76be0abf048863342c90e15d1aadf150b8d7c5212e5fc391edd8 I0312 16:07:09.784446 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=afda28928cfe505cea3eec6e6f8102c9f32dd586611e75b3e270c3f213aaebef I0312 16:07:09.784476 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=0c57da8b23c1d34d4cad02e6e7747b3a14e7bcb25279e89f4678168fe3cd835a I0312 16:07:09.784522 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=7a61196f4baad74129c84ec540bd90cebfdafc74420bdaee95553af0e41b0258 I0312 16:07:09.784540 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=882aaa26687d682620b8fc312a007ed39c9f7c91a92adb6ffb151c4599c75436 I0312 16:07:09.784553 1 recorder.go:70] Recording events/openshift-kube-storage-version-migrator with fingerprint=5b962a12d34cb8abd1522ccd18974746a521b214ff18085c083cb43dfd6e0845 I0312 16:07:09.784603 1 recorder.go:70] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=f8889efd99b3115aabe6a4ffb38d67dc67abb407c3c770c92e8717b7a562365b I0312 16:07:09.784734 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-77ng7 with fingerprint=e17d3ff8e6c3c4a732463b95a8ffa54592d5f9041c3251c7029be1abc60d8c56 I0312 16:07:09.784817 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-slbd7 with fingerprint=6fff88282d38d088b759f027139876eac177ab4279d84b4dd0810d32c351d02f I0312 16:07:09.784887 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-xtptc with fingerprint=87006a38ddb4f1ab4206c8c7d5c642627d42d473390f2ca2021284ad91e893c3 I0312 16:07:09.784992 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-6cb9fd6766-p4zcc with fingerprint=7c440c3eae8262d9386e2ab50dce8ba5aeaedc2b659e80b9cc895471a288d054 I0312 16:07:09.785084 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-7776cbcf7c-lkjjg with fingerprint=bc9edbc0358606f0ee9ff0f8973eff6e85ddec537fcc7c73bec750b5d863b6a8 I0312 16:07:09.785172 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-7776cbcf7c-svgj7 with fingerprint=a9fb4b86ff01028e9549b6eeeceb5bc10c8087c116a8af08663398a99cc9e40c I0312 16:07:09.785243 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-599vg with fingerprint=dd2b998ec2d6daf8bee455f3db55f1948b3212d24f7200867bca15fb2b875510 I0312 16:07:09.785297 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-knlng with fingerprint=acd77ffdabd1f81de315d4f7c8ff5e4c63626d5126f44d6e561e01ec526b005b I0312 16:07:09.785352 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-wn647 with fingerprint=93cdd72870740f6d18718b67060b4e437d1b7ff3c3c7d7f25bc17d0a9a787adf I0312 16:07:09.785369 1 recorder.go:70] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-949fdd477-sxvz5/migrator_current.log with fingerprint=5588fa40c5a636ef4e6f729f8465cc68637fe2b7a6ce96ea6831533e34126593 I0312 16:07:09.785376 1 recorder.go:70] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-949fdd477-sxvz5/graceful-termination_current.log with fingerprint=927e189d5beb50cb3f659694b27a7e4b8ad5925ef0178f6ed0c084caf84209c8 I0312 16:07:09.785442 1 recorder.go:70] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-7944b9f44d-smpwz/kube-storage-version-migrator-operator_current.log with fingerprint=98894c43bec8555b38d055b5f8149987001c9dc5f53c50b6ab87582e5af88ce1 I0312 16:07:09.785455 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.064149605s to process 19 records I0312 16:07:15.797293 1 configmapobserver.go:84] configmaps "insights-config" not found I0312 16:07:16.445331 1 tasks_processing.go:74] worker 10 stopped. I0312 16:07:16.445373 1 recorder.go:70] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0312 16:07:16.445391 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.700890976s to process 1 records I0312 16:07:17.005657 1 configmapobserver.go:84] configmaps "insights-config" not found I0312 16:07:17.125810 1 tasks_processing.go:74] worker 15 stopped. I0312 16:07:17.126097 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=bf87f2531cc0884b07268060c8c965a23390b0089a10c3b066309a34832b594a I0312 16:07:17.126117 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.405511856s to process 1 records E0312 16:07:17.126176 1 periodic.go:250] "Unhandled Error" err="clusterconfig failed after 13.408s with: function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_configs\" failed with an error, function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0312 16:07:17.127250 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_configs" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0312 16:07:17.127266 1 periodic.go:212] Running workloads gatherer I0312 16:07:17.127287 1 tasks_processing.go:45] number of workers: 2 I0312 16:07:17.127303 1 tasks_processing.go:69] worker 1 listening for tasks. I0312 16:07:17.127311 1 tasks_processing.go:71] worker 1 working on workload_info task. I0312 16:07:17.127315 1 tasks_processing.go:69] worker 0 listening for tasks. I0312 16:07:17.127386 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0312 16:07:17.150317 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0312 16:07:17.152931 1 tasks_processing.go:74] worker 0 stopped. I0312 16:07:17.152945 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 25.530181ms to process 0 records I0312 16:07:17.158985 1 gather_workloads_info.go:387] No image sha256:a7c71f3c9714cf1717c45e12ec817be6d6209f00989e6a31d634e527f0c8147d (9ms) I0312 16:07:17.167907 1 gather_workloads_info.go:387] No image sha256:bae7a33f8db8a3d4b3c4c05498aba85a0ce463f85322067c48e79663710e616e (9ms) I0312 16:07:17.175886 1 gather_workloads_info.go:387] No image sha256:8ef4bdf07dda423fab73484dbc66d527ce8ba8cc2d4f99210bc1bc24ea08c0cb (8ms) I0312 16:07:17.184064 1 gather_workloads_info.go:387] No image sha256:7941b5d9a758b8667e99cd9236f7f96ec036af61df5cbc96e16077d36700d7c7 (8ms) I0312 16:07:17.191881 1 gather_workloads_info.go:387] No image sha256:d28734effaadd66434e77a1a5fbe2e8a4ca2066cd9f8868c22ade9475539bfd7 (8ms) I0312 16:07:17.202258 1 gather_workloads_info.go:387] No image sha256:3d0634fe58641d5242649c44ddba70ed67fdd6d2dcb5c2261df5cee8b33de9fd (10ms) I0312 16:07:17.205173 1 configmapobserver.go:84] configmaps "insights-config" not found I0312 16:07:17.210053 1 gather_workloads_info.go:387] No image sha256:18956d5fbc8d0369a26cb1e00eb5c3aff1973ae8cbd3983920a0cf2851f05358 (8ms) I0312 16:07:17.217812 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (8ms) I0312 16:07:17.225687 1 gather_workloads_info.go:387] No image sha256:23c4fe84047a4e6cbe7f75f470e7d6fe0e61e7910dd17945b1b61bc4b72f3f2a (8ms) I0312 16:07:17.233868 1 gather_workloads_info.go:387] No image sha256:9697bc2258bdfa9ae8c1866cc7eb0b3b46851998db827b121bdf77417a881eb3 (8ms) I0312 16:07:17.258801 1 gather_workloads_info.go:387] No image sha256:824a3b8f78e19aa21a9f6444aefe8e0b624886cca18fe828b13acbab55e6e868 (25ms) I0312 16:07:17.359422 1 gather_workloads_info.go:387] No image sha256:420c12f61c53e54eab2d99476759c0de339fca98ce0a3a782bc6545cc0e97a9c (101ms) I0312 16:07:17.459403 1 gather_workloads_info.go:387] No image sha256:8b1e32ce43eb1247b0558230b6e0baa85ac62f1a5e2a2089d40b4fea90538529 (100ms) I0312 16:07:17.559379 1 gather_workloads_info.go:387] No image sha256:752be69b2262be713df12c47f4bac8c2dafed272c401e9a89f8060f053d68054 (100ms) I0312 16:07:17.659036 1 gather_workloads_info.go:387] No image sha256:2d077e73dd76873ee2a1583aa3b3e76a1b408737f0af2e22b2aa055604d89e81 (100ms) I0312 16:07:17.759114 1 gather_workloads_info.go:387] No image sha256:4f175ee49f51dc4379e9993fecd1657c7a9c4c3dc096772cb198d0212b9eea47 (100ms) I0312 16:07:17.859396 1 gather_workloads_info.go:387] No image sha256:0a92dd43975f972e3dc707fb37854e046813531553038147e9c90d54a8d9df73 (100ms) I0312 16:07:17.959597 1 gather_workloads_info.go:387] No image sha256:6d6ebff54b8adac74f4d1b12ac8aa0f16cd7b28370e0d6aa847d1c457e03a5b6 (100ms) I0312 16:07:18.058871 1 gather_workloads_info.go:387] No image sha256:48404bd61d05dc738ccce4d22e36b30dbcdc6015b06b4afb604fd1baeee35bf2 (99ms) I0312 16:07:18.158823 1 gather_workloads_info.go:387] No image sha256:f2c46054a8f64a0e949cbf30295f31b4a35a0203c6ad03fa7ec922b4101dcbf8 (100ms) I0312 16:07:18.258754 1 gather_workloads_info.go:387] No image sha256:373614619b9420b110d1508c5f17e066ce69c4c226fd04d02fbb959d9ba41eb6 (100ms) I0312 16:07:18.358859 1 gather_workloads_info.go:387] No image sha256:ceab8a8c340801fcebf800620d3fef5493d8a443077a21c6e56984051bd3abde (100ms) I0312 16:07:18.458986 1 gather_workloads_info.go:387] No image sha256:ccdda82b4adfa8ee4a84b3bad19693be6a42a9881a7b83788d835912129c49a4 (100ms) I0312 16:07:18.559101 1 gather_workloads_info.go:387] No image sha256:87efe06afd1f04426fca7f86c0f74c4ee75c311ba199ddabd5b849b877bc59fa (100ms) I0312 16:07:18.660098 1 gather_workloads_info.go:387] No image sha256:14703f73bb1ca69ea03b726e340b3b68f8e294e1f22c80d1a16666ea3d4a88a3 (101ms) I0312 16:07:18.759720 1 gather_workloads_info.go:387] No image sha256:944d9261ba7a143131fe8267c172defc5f37acc1cea3d4d373ec6fc5d8bfcc31 (100ms) I0312 16:07:18.859125 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (99ms) I0312 16:07:18.959387 1 gather_workloads_info.go:387] No image sha256:76f820bd9bd138d29305d545d7d49bfe63e2923f1fb1ec2e8eac81a388359024 (100ms) I0312 16:07:19.059975 1 gather_workloads_info.go:387] No image sha256:0a8a0473029ace3adbb66f490dbe560f0e7782a38cdf32c1f7dd3e092e1d191e (101ms) I0312 16:07:19.159394 1 gather_workloads_info.go:387] No image sha256:2fac754deaeade3456361eed52e344318ff16d04819384432759f0ea35530114 (99ms) I0312 16:07:19.259721 1 gather_workloads_info.go:387] No image sha256:6e67c980b7300e769fb4b2adaaf006d0f8274e43b10586701b204ea5153f15fc (100ms) I0312 16:07:19.358993 1 gather_workloads_info.go:387] No image sha256:a9e885d6f0456a2cc10f9e5da71fe9403f5e4c639b9b7ea15bd403b272ccb824 (99ms) I0312 16:07:19.359020 1 tasks_processing.go:74] worker 1 stopped. E0312 16:07:19.359037 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0312 16:07:19.359326 1 recorder.go:70] Recording config/workload_info with fingerprint=199a7e83f9dd26a6b2b3114a2e3c32bcc4280336ebdcbf262daf98e71a7ef6bf I0312 16:07:19.359351 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.231702118s to process 1 records E0312 16:07:19.359383 1 periodic.go:250] "Unhandled Error" err="workloads failed after 2.232s with: function \"workload_info\" failed with an error" I0312 16:07:19.360488 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0312 16:07:19.360501 1 periodic.go:212] Running conditional gatherer I0312 16:07:19.368461 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules I0312 16:07:19.375504 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.9:53358->172.30.0.10:53: read: connection refused E0312 16:07:19.375790 1 conditional_gatherer.go:320] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0312 16:07:19.375847 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0312 16:07:19.383607 1 conditional_gatherer.go:390] cluster version is '4.19.9' E0312 16:07:19.383621 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383626 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383630 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383634 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383637 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383640 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383644 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383647 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0312 16:07:19.383650 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0312 16:07:19.383663 1 tasks_processing.go:45] number of workers: 3 I0312 16:07:19.383675 1 tasks_processing.go:69] worker 2 listening for tasks. I0312 16:07:19.383679 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0312 16:07:19.383692 1 tasks_processing.go:69] worker 0 listening for tasks. I0312 16:07:19.383703 1 tasks_processing.go:69] worker 1 listening for tasks. I0312 16:07:19.383713 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0312 16:07:19.383731 1 tasks_processing.go:74] worker 1 stopped. I0312 16:07:19.383735 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0312 16:07:19.383780 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0312 16:07:19.383797 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 805ns to process 1 records I0312 16:07:19.383831 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0312 16:07:19.383847 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.216µs to process 1 records I0312 16:07:19.383849 1 tasks_processing.go:74] worker 0 stopped. I0312 16:07:19.383984 1 tasks_processing.go:74] worker 2 stopped. I0312 16:07:19.383996 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 260.71µs to process 0 records I0312 16:07:19.384014 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.9:53358->172.30.0.10:53: read: connection refused I0312 16:07:19.384031 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0312 16:07:19.405606 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=4a9225d95bb6e303abcc578c8789da71ac53485c46e60f625dcb51572670f0af I0312 16:07:19.405758 1 diskrecorder.go:70] Writing 111 records to /var/lib/insights-operator/insights-2026-03-12-160719.tar.gz I0312 16:07:19.415052 1 diskrecorder.go:51] Wrote 111 records to disk in 9ms I0312 16:07:19.415087 1 periodic.go:281] Gathering cluster info every 2h0m0s I0312 16:07:19.415102 1 periodic.go:282] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0312 16:07:26.298961 1 configmapobserver.go:84] configmaps "insights-config" not found I0312 16:08:23.260044 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="70e8a2fc2a71a9b8710e79ea3d59733afccb0603b64c163a5ba3cfd12d164f29") W0312 16:08:23.260079 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0312 16:08:23.260131 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="0b4c965dc5f48c20ab6361c929bbf1df18fc59da27d7801050b5e114a37a5629")