W0311 06:24:35.558241 1 cmd.go:257] Using insecure, self-signed certificates I0311 06:24:35.891232 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 06:24:35.891547 1 observer_polling.go:159] Starting file observer W0311 06:24:35.913727 1 builder.go:272] unable to get owner reference (falling back to namespace): replicasets.apps "insights-operator-5ff5cb4f99" is forbidden: User "system:serviceaccount:openshift-insights:operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-insights" I0311 06:24:36.062090 1 operator.go:59] Starting insights-operator v0.0.0-master+$Format:%H$ I0311 06:24:36.062322 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0311 06:24:36.062880 1 secure_serving.go:57] Forcing use of http/1.1 only W0311 06:24:36.062901 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0311 06:24:36.062905 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0311 06:24:36.062911 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0311 06:24:36.062915 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0311 06:24:36.062917 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0311 06:24:36.062920 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0311 06:24:36.063068 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0311 06:24:36.067068 1 operator.go:124] FeatureGates initialized: knownFeatureGates=[AWSEFSDriverVolumeMetrics AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BareMetalLoadBalancer BuildCSIVolumes CPMSMachineNamePrefix ChunkSizeMiB CloudDualStackNodeIPs ConsolePluginContentSecurityPolicy DisableKubeletCloudCredentialProviders ExternalOIDC GCPLabelsTags GatewayAPI GatewayAPIController HardwareSpeed IngressControllerLBSubnetsAWS KMSv1 ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles MultiArchInstallAWS MultiArchInstallGCP NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation NodeDisruptionPolicy OnClusterBuild PersistentIPsForVirtualization PrivateHostedZoneAWS RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController VSphereDriverConfiguration VSphereMultiVCenters ValidatingAdmissionPolicy AWSClusterHostedDNS AutomatedEtcdBackup BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNS GCPCustomAPIEndpoints HighlyAvailableArbiter ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather InsightsRuntimeExtractor KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController MachineAPIProviderOpenStack MachineConfigNodes MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PinnedImages PlatformOperators ProcMountType RouteAdvertisements SELinuxChangePolicy SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerification SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass VolumeGroupSnapshot] I0311 06:24:36.067125 1 event.go:377] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"openshift-insights", Name:"openshift-insights", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AWSEFSDriverVolumeMetrics", "AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BareMetalLoadBalancer", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ChunkSizeMiB", "CloudDualStackNodeIPs", "ConsolePluginContentSecurityPolicy", "DisableKubeletCloudCredentialProviders", "ExternalOIDC", "GCPLabelsTags", "GatewayAPI", "GatewayAPIController", "HardwareSpeed", "IngressControllerLBSubnetsAWS", "KMSv1", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "MultiArchInstallAWS", "MultiArchInstallGCP", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NodeDisruptionPolicy", "OnClusterBuild", "PersistentIPsForVirtualization", "PrivateHostedZoneAWS", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "VSphereDriverConfiguration", "VSphereMultiVCenters", "ValidatingAdmissionPolicy"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AutomatedEtcdBackup", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "HighlyAvailableArbiter", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "InsightsRuntimeExtractor", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "MachineAPIProviderOpenStack", "MachineConfigNodes", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PinnedImages", "PlatformOperators", "ProcMountType", "RouteAdvertisements", "SELinuxChangePolicy", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0311 06:24:36.067571 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0311 06:24:36.067628 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0311 06:24:36.067639 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0311 06:24:36.067585 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0311 06:24:36.067706 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0311 06:24:36.067694 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0311 06:24:36.067876 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3406083701/tls.crt::/tmp/serving-cert-3406083701/tls.key" I0311 06:24:36.068105 1 secure_serving.go:213] Serving securely on [::]:8443 I0311 06:24:36.068135 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" E0311 06:24:36.070248 1 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:serviceaccount:openshift-insights:operator\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-insights\"" event="&Event{ObjectMeta:{openshift-insights.189bb559e90c259e openshift-insights 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Namespace,Namespace:openshift-insights,Name:openshift-insights,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{\"AWSEFSDriverVolumeMetrics\", \"AdditionalRoutingCapabilities\", \"AdminNetworkPolicy\", \"AlibabaPlatform\", \"AzureWorkloadIdentity\", \"BareMetalLoadBalancer\", \"BuildCSIVolumes\", \"CPMSMachineNamePrefix\", \"ChunkSizeMiB\", \"CloudDualStackNodeIPs\", \"ConsolePluginContentSecurityPolicy\", \"DisableKubeletCloudCredentialProviders\", \"ExternalOIDC\", \"GCPLabelsTags\", \"GatewayAPI\", \"GatewayAPIController\", \"HardwareSpeed\", \"IngressControllerLBSubnetsAWS\", \"KMSv1\", \"ManagedBootImages\", \"ManagedBootImagesAWS\", \"MetricsCollectionProfiles\", \"MultiArchInstallAWS\", \"MultiArchInstallGCP\", \"NetworkDiagnosticsConfig\", \"NetworkLiveMigration\", \"NetworkSegmentation\", \"NodeDisruptionPolicy\", \"OnClusterBuild\", \"PersistentIPsForVirtualization\", \"PrivateHostedZoneAWS\", \"RouteExternalCertificate\", \"ServiceAccountTokenNodeBinding\", \"SetEIPForNLBIngressController\", \"VSphereDriverConfiguration\", \"VSphereMultiVCenters\", \"ValidatingAdmissionPolicy\"}, Disabled:[]v1.FeatureGateName{\"AWSClusterHostedDNS\", \"AutomatedEtcdBackup\", \"BootcNodeManagement\", \"ClusterAPIInstall\", \"ClusterAPIInstallIBMCloud\", \"ClusterMonitoringConfig\", \"ClusterVersionOperatorConfiguration\", \"DNSNameResolver\", \"DualReplica\", \"DyanmicServiceEndpointIBMCloud\", \"DynamicResourceAllocation\", \"EtcdBackendQuota\", \"EventedPLEG\", \"Example\", \"Example2\", \"ExternalOIDCWithUIDAndExtraClaimMappings\", \"GCPClusterHostedDNS\", \"GCPCustomAPIEndpoints\", \"HighlyAvailableArbiter\", \"ImageStreamImportMode\", \"IngressControllerDynamicConfigurationManager\", \"InsightsConfig\", \"InsightsConfigAPI\", \"InsightsOnDemandDataGather\", \"InsightsRuntimeExtractor\", \"KMSEncryptionProvider\", \"MachineAPIMigration\", \"MachineAPIOperatorDisableMachineHealthCheckController\", \"MachineAPIProviderOpenStack\", \"MachineConfigNodes\", \"MaxUnavailableStatefulSet\", \"MinimumKubeletVersion\", \"MixedCPUsAllocation\", \"MultiArchInstallAzure\", \"NewOLM\", \"NewOLMCatalogdAPIV1Metas\", \"NewOLMOwnSingleNamespace\", \"NewOLMPreflightPermissionChecks\", \"NodeSwap\", \"NutanixMultiSubnets\", \"OVNObservability\", \"OpenShiftPodSecurityAdmission\", \"PinnedImages\", \"PlatformOperators\", \"ProcMountType\", \"RouteAdvertisements\", \"SELinuxChangePolicy\", \"SELinuxMount\", \"ShortCertRotation\", \"SignatureStores\", \"SigstoreImageVerification\", \"SigstoreImageVerificationPKI\", \"TranslateStreamCloseWebsocketRequests\", \"UpgradeStatus\", \"UserNamespacesPodSecurityStandards\", \"UserNamespacesSupport\", \"VSphereConfigurableMaxAllowedBlockVolumesPerNode\", \"VSphereHostVMGroupZonal\", \"VSphereMultiDisk\", \"VSphereMultiNetworks\", \"VolumeAttributesClass\", \"VolumeGroupSnapshot\"}},Source:EventSource{Component:openshift-insights-operator,Host:,},FirstTimestamp:2026-03-11 06:24:36.067026334 +0000 UTC m=+0.550324352,LastTimestamp:2026-03-11 06:24:36.067026334 +0000 UTC m=+0.550324352,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:openshift-insights-operator,ReportingInstance:,}" W0311 06:24:36.072063 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0311 06:24:36.072086 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0311 06:24:36.072197 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0311 06:24:36.076505 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0311 06:24:36.076527 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0311 06:24:36.080860 1 secretconfigobserver.go:119] support secret does not exist I0311 06:24:36.085056 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0311 06:24:36.088757 1 secretconfigobserver.go:119] support secret does not exist I0311 06:24:36.092082 1 recorder.go:156] Pruning old reports every 7h45m56s, max age is 288h0m0s I0311 06:24:36.096293 1 periodic.go:212] Running clusterconfig gatherer I0311 06:24:36.096309 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0311 06:24:36.096327 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0311 06:24:36.096337 1 tasks_processing.go:45] number of workers: 32 I0311 06:24:36.096350 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0311 06:24:36.096357 1 insightsreport.go:296] Starting report retriever I0311 06:24:36.096362 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0311 06:24:36.096375 1 tasks_processing.go:69] worker 19 listening for tasks. I0311 06:24:36.096375 1 tasks_processing.go:69] worker 9 listening for tasks. I0311 06:24:36.096383 1 tasks_processing.go:71] worker 19 working on machines task. I0311 06:24:36.096387 1 tasks_processing.go:69] worker 14 listening for tasks. I0311 06:24:36.096386 1 tasks_processing.go:69] worker 26 listening for tasks. I0311 06:24:36.096388 1 tasks_processing.go:69] worker 0 listening for tasks. I0311 06:24:36.096392 1 tasks_processing.go:69] worker 25 listening for tasks. I0311 06:24:36.096403 1 tasks_processing.go:69] worker 28 listening for tasks. I0311 06:24:36.096402 1 tasks_processing.go:69] worker 2 listening for tasks. I0311 06:24:36.096394 1 tasks_processing.go:69] worker 27 listening for tasks. I0311 06:24:36.096411 1 tasks_processing.go:69] worker 3 listening for tasks. I0311 06:24:36.096413 1 tasks_processing.go:69] worker 20 listening for tasks. I0311 06:24:36.096395 1 tasks_processing.go:69] worker 1 listening for tasks. I0311 06:24:36.096421 1 tasks_processing.go:69] worker 4 listening for tasks. I0311 06:24:36.096404 1 tasks_processing.go:69] worker 10 listening for tasks. I0311 06:24:36.096426 1 tasks_processing.go:69] worker 30 listening for tasks. I0311 06:24:36.096428 1 tasks_processing.go:69] worker 13 listening for tasks. I0311 06:24:36.096411 1 tasks_processing.go:69] worker 11 listening for tasks. I0311 06:24:36.096419 1 tasks_processing.go:69] worker 29 listening for tasks. I0311 06:24:36.096436 1 tasks_processing.go:69] worker 5 listening for tasks. I0311 06:24:36.096436 1 tasks_processing.go:69] worker 17 listening for tasks. I0311 06:24:36.096440 1 tasks_processing.go:69] worker 23 listening for tasks. I0311 06:24:36.096435 1 tasks_processing.go:69] worker 22 listening for tasks. I0311 06:24:36.096422 1 tasks_processing.go:69] worker 12 listening for tasks. I0311 06:24:36.096429 1 tasks_processing.go:69] worker 21 listening for tasks. I0311 06:24:36.096443 1 tasks_processing.go:69] worker 7 listening for tasks. I0311 06:24:36.096448 1 tasks_processing.go:69] worker 8 listening for tasks. I0311 06:24:36.096451 1 tasks_processing.go:69] worker 6 listening for tasks. I0311 06:24:36.096454 1 tasks_processing.go:69] worker 24 listening for tasks. I0311 06:24:36.096443 1 tasks_processing.go:69] worker 15 listening for tasks. I0311 06:24:36.096451 1 tasks_processing.go:69] worker 16 listening for tasks. I0311 06:24:36.096459 1 tasks_processing.go:69] worker 18 listening for tasks. I0311 06:24:36.096375 1 tasks_processing.go:69] worker 31 listening for tasks. I0311 06:24:36.096528 1 tasks_processing.go:71] worker 0 working on support_secret task. I0311 06:24:36.096535 1 tasks_processing.go:71] worker 31 working on certificate_signing_requests task. I0311 06:24:36.096538 1 tasks_processing.go:71] worker 28 working on validating_webhook_configurations task. I0311 06:24:36.096542 1 tasks_processing.go:71] worker 14 working on version task. I0311 06:24:36.096555 1 tasks_processing.go:71] worker 26 working on dvo_metrics task. I0311 06:24:36.096631 1 tasks_processing.go:71] worker 9 working on ingress task. I0311 06:24:36.096640 1 tasks_processing.go:71] worker 17 working on image_pruners task. I0311 06:24:36.096719 1 tasks_processing.go:71] worker 10 working on ingress_certificates task. I0311 06:24:36.096838 1 tasks_processing.go:71] worker 30 working on container_images task. I0311 06:24:36.096529 1 tasks_processing.go:71] worker 27 working on authentication task. I0311 06:24:36.097078 1 tasks_processing.go:71] worker 23 working on ceph_cluster task. I0311 06:24:36.097119 1 tasks_processing.go:71] worker 21 working on storage_classes task. I0311 06:24:36.097177 1 tasks_processing.go:71] worker 7 working on openstack_dataplanedeployments task. I0311 06:24:36.097191 1 tasks_processing.go:71] worker 2 working on sap_datahubs task. I0311 06:24:36.097313 1 tasks_processing.go:71] worker 5 working on infrastructures task. I0311 06:24:36.097354 1 tasks_processing.go:71] worker 1 working on olm_operators task. I0311 06:24:36.097407 1 tasks_processing.go:71] worker 4 working on monitoring_persistent_volumes task. I0311 06:24:36.097090 1 tasks_processing.go:71] worker 11 working on node_logs task. I0311 06:24:36.097320 1 tasks_processing.go:71] worker 18 working on networks task. I0311 06:24:36.097095 1 tasks_processing.go:71] worker 13 working on nodenetworkstates task. I0311 06:24:36.097101 1 tasks_processing.go:71] worker 29 working on oauths task. I0311 06:24:36.097107 1 tasks_processing.go:71] worker 6 working on operators_pods_and_events task. I0311 06:24:36.097112 1 tasks_processing.go:71] worker 22 working on config_maps task. I0311 06:24:36.097116 1 tasks_processing.go:71] worker 12 working on overlapping_namespace_uids task. I0311 06:24:36.097325 1 tasks_processing.go:71] worker 24 working on openshift_logging task. I0311 06:24:36.097329 1 tasks_processing.go:71] worker 15 working on silenced_alerts task. W0311 06:24:36.098243 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 06:24:36.098259 1 tasks_processing.go:71] worker 15 working on crds task. I0311 06:24:36.097333 1 tasks_processing.go:71] worker 16 working on clusterroles task. I0311 06:24:36.097339 1 tasks_processing.go:71] worker 20 working on nodenetworkconfigurationpolicies task. I0311 06:24:36.097343 1 tasks_processing.go:71] worker 3 working on active_alerts task. W0311 06:24:36.098488 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 06:24:36.097345 1 tasks_processing.go:71] worker 8 working on openshift_machine_api_events task. I0311 06:24:36.096534 1 tasks_processing.go:71] worker 25 working on openstack_dataplanenodesets task. I0311 06:24:36.098359 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 35.401µs to process 0 records I0311 06:24:36.098635 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 36.149µs to process 0 records I0311 06:24:36.098658 1 tasks_processing.go:71] worker 3 working on openstack_version task. I0311 06:24:36.099516 1 tasks_processing.go:71] worker 19 working on machine_config_pools task. E0311 06:24:36.099529 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0311 06:24:36.099546 1 gather.go:177] gatherer "clusterconfig" function "machines" took 3.120728ms to process 0 records I0311 06:24:36.111418 1 controller.go:119] Initializing last reported time to 0001-01-01T00:00:00Z I0311 06:24:36.111440 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0311 06:24:36.111447 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0311 06:24:36.111451 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0311 06:24:36.111464 1 controller.go:458] The operator is still being initialized I0311 06:24:36.111469 1 controller.go:481] The operator is healthy I0311 06:24:36.111612 1 tasks_processing.go:71] worker 9 working on install_plans task. I0311 06:24:36.111878 1 recorder.go:70] Recording config/ingress with fingerprint=a836405a05c91b6ee61fcc6becedb0f59528f8c092b1b262a3b8330a0503f26c I0311 06:24:36.111895 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 14.964228ms to process 1 records I0311 06:24:36.113306 1 tasks_processing.go:71] worker 23 working on feature_gates task. I0311 06:24:36.113320 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 16.182475ms to process 0 records I0311 06:24:36.113754 1 tasks_processing.go:71] worker 7 working on machine_healthchecks task. I0311 06:24:36.113772 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 16.563045ms to process 0 records I0311 06:24:36.113982 1 tasks_processing.go:71] worker 17 working on pdbs task. I0311 06:24:36.116849 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=68760367308bd4a4693218fb08553a881c9b2c919f246ef85594a5daf64bc9cf I0311 06:24:36.116875 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 17.154561ms to process 1 records I0311 06:24:36.116992 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 16.605778ms to process 0 records E0311 06:24:36.117010 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0311 06:24:36.117022 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 18.876457ms to process 0 records I0311 06:24:36.117033 1 tasks_processing.go:71] worker 0 working on image_registries task. I0311 06:24:36.117145 1 tasks_processing.go:71] worker 2 working on schedulers task. I0311 06:24:36.117207 1 tasks_processing.go:71] worker 27 working on storage_cluster task. I0311 06:24:36.117957 1 recorder.go:70] Recording config/authentication with fingerprint=77841ef6b7265808812ede2bd3f8b45f49f3ce20ac01ddcb5c22af811e9ee4e4 I0311 06:24:36.117975 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 20.270769ms to process 1 records I0311 06:24:36.117996 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 19.531239ms to process 0 records I0311 06:24:36.118004 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 18.873183ms to process 0 records I0311 06:24:36.118082 1 tasks_processing.go:71] worker 21 working on machine_autoscalers task. I0311 06:24:36.118142 1 tasks_processing.go:71] worker 20 working on cluster_apiserver task. I0311 06:24:36.118149 1 tasks_processing.go:71] worker 13 working on jaegers task. I0311 06:24:36.119329 1 recorder.go:70] Recording config/storage/storageclasses/gp2-csi with fingerprint=4dcbe05f6ee0ff4e357441af6fd22b13df1d2a782fb11344dc4ec71ed2cdaa8a I0311 06:24:36.119407 1 recorder.go:70] Recording config/storage/storageclasses/gp3-csi with fingerprint=2659329ac4b16fa197d314aa3dde666cb5d4dc22d68dd05632daef133208f2a6 I0311 06:24:36.119445 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 20.376263ms to process 2 records I0311 06:24:36.119491 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 18.992144ms to process 0 records I0311 06:24:36.119520 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 20.26227ms to process 0 records I0311 06:24:36.119545 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 19.198749ms to process 0 records I0311 06:24:36.119745 1 tasks_processing.go:71] worker 4 working on service_accounts task. I0311 06:24:36.119879 1 recorder.go:70] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=198de867f9bb7d85d3b97e9f1324a487ca9cec755383294083b832f8e88c4ebf I0311 06:24:36.119992 1 tasks_processing.go:71] worker 3 working on openstack_controlplanes task. I0311 06:24:36.120124 1 recorder.go:70] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=98e1c9a1342fb1b2ef7efa69f8f2dbda697f3fad1823552d4d8dd295f3a86762 I0311 06:24:36.120166 1 recorder.go:70] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=abf65552d26240f053809c94d2015bdb2bf6b5c6521bff0a151d72a3de513e0a I0311 06:24:36.120274 1 tasks_processing.go:71] worker 28 working on cost_management_metrics_configs task. I0311 06:24:36.120309 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=86c0207f933a756f13c924b493db3adbd15ac42ccb8627bedfa8c36d53fa2e31 I0311 06:24:36.120417 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=a42fa09b239ee0c88bec06ad8f56c8aa63950bc531a4646a5f2a694ffeb398db I0311 06:24:36.120471 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=3a4e78448ebde5fb190b7f10e6b02f8329ac349814380d3a3109cfb83d498fb6 I0311 06:24:36.120172 1 tasks_processing.go:71] worker 25 working on machine_sets task. I0311 06:24:36.120602 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=17c3f38e8abaae9b275aa2b717ff28e1dde54892e571af8bc1756540ce188df9 I0311 06:24:36.120922 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=99310436e929f0f0dd3bc8df2c9aa513b462e3a6fd6f37dbcf95fef91f2d433b I0311 06:24:36.121084 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=001e711ccf543d3316b597478ddba0c07a12f3593e9133663281525a12f3741b I0311 06:24:36.121232 1 recorder.go:70] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=afe272d7921e4f8ad7ce6aa4bb294d71cb9b507f1bea647230f9a652c9aeaf85 I0311 06:24:36.121311 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 21.279134ms to process 10 records I0311 06:24:36.121638 1 recorder.go:70] Recording config/network with fingerprint=1548d4ae73cd39aeeeeca4ac6f26571ab01f894758e71008016333f51c0c33f8 I0311 06:24:36.121716 1 gather.go:177] gatherer "clusterconfig" function "networks" took 20.241594ms to process 1 records I0311 06:24:36.121750 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 21.846624ms to process 0 records I0311 06:24:36.121786 1 tasks_processing.go:71] worker 18 working on sap_config task. I0311 06:24:36.121905 1 tasks_processing.go:71] worker 11 working on operators task. I0311 06:24:36.122110 1 tasks_processing.go:71] worker 29 working on sap_pods task. I0311 06:24:36.122341 1 recorder.go:70] Recording config/oauth with fingerprint=fcb1bd6f4c02948dfb56111218ba925d67d30765afce190d6e0fe2f73d0c441a I0311 06:24:36.122431 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 21.563693ms to process 1 records I0311 06:24:36.122806 1 tasks_processing.go:71] worker 5 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0311 06:24:36.123857 1 recorder.go:70] Recording config/infrastructure with fingerprint=a97a75240ab6d8561e3bd1b7843bde941120a6f8d525629a5dbbdcd589d830b6 I0311 06:24:36.123925 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 22.016266ms to process 1 records I0311 06:24:36.123940 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 21.510819ms to process 0 records I0311 06:24:36.123946 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 21.286949ms to process 0 records I0311 06:24:36.123952 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 3.140112ms to process 0 records E0311 06:24:36.123961 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0311 06:24:36.123974 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 6.717475ms to process 0 records I0311 06:24:36.123981 1 tasks_processing.go:71] worker 24 working on proxies task. I0311 06:24:36.123974 1 tasks_processing.go:71] worker 8 working on machine_configs task. I0311 06:24:36.124043 1 tasks_processing.go:71] worker 19 working on mutating_webhook_configurations task. I0311 06:24:36.124238 1 tasks_processing.go:71] worker 7 working on pod_network_connectivity_checks task. I0311 06:24:36.124005 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 22.256574ms to process 0 records I0311 06:24:36.124438 1 tasks_processing.go:71] worker 27 working on tsdb_status task. I0311 06:24:36.124445 1 tasks_processing.go:71] worker 31 working on lokistack task. W0311 06:24:36.124466 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory W0311 06:24:36.124472 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 06:24:36.124437 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 25.352528ms to process 0 records I0311 06:24:36.124510 1 recorder.go:70] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0311 06:24:36.124529 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 23.846049ms to process 1 records I0311 06:24:36.124647 1 tasks_processing.go:71] worker 12 working on metrics task. I0311 06:24:36.124649 1 tasks_processing.go:71] worker 23 working on aggregated_monitoring_cr_names task. W0311 06:24:36.124687 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 06:24:36.124714 1 recorder.go:70] Recording config/featuregate with fingerprint=d0e9cb8f139b6d74b772e1c7d8a2d55609c4398e81eb700443bc7723374ca984 I0311 06:24:36.124754 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 9.419915ms to process 1 records I0311 06:24:36.124787 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 5.731817ms to process 0 records I0311 06:24:36.124796 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 5.493449ms to process 0 records I0311 06:24:36.124805 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 33.198µs to process 0 records I0311 06:24:36.124815 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 34.758µs to process 0 records I0311 06:24:36.124825 1 tasks_processing.go:71] worker 12 working on container_runtime_configs task. I0311 06:24:36.124871 1 tasks_processing.go:74] worker 27 stopped. I0311 06:24:36.124875 1 tasks_processing.go:71] worker 21 working on image task. I0311 06:24:36.124878 1 tasks_processing.go:71] worker 13 working on nodes task. I0311 06:24:36.125745 1 tasks_processing.go:74] worker 2 stopped. I0311 06:24:36.127797 1 recorder.go:70] Recording config/schedulers/cluster with fingerprint=4b4937b52b6d6e10bc0afc918ffc02612f098c889a757815bf3d94154be706ac I0311 06:24:36.127817 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 8.54763ms to process 1 records I0311 06:24:36.128071 1 tasks_processing.go:74] worker 17 stopped. I0311 06:24:36.128501 1 recorder.go:70] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=d0706ef73745337e5e2a5bcb34edbdbe21571ec3feb7b20c34bc344c5c2204d7 I0311 06:24:36.128525 1 recorder.go:70] Recording config/pdbs/openshift-ingress/router-default with fingerprint=15f26579bf5477e0d4dd37277f36114f6d4f1373d2a9ac21bed06b809741c18f I0311 06:24:36.128543 1 recorder.go:70] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=295cfe77dfcaa6ae65e2f535bf5632fb4ea9425edd44e2f3374bf4a2512fd6d7 I0311 06:24:36.128552 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 14.068095ms to process 3 records I0311 06:24:36.128638 1 tasks_processing.go:74] worker 20 stopped. I0311 06:24:36.128837 1 recorder.go:70] Recording config/apiserver with fingerprint=b3739981fdce58226e5f4b95c33e7eb8037e6bc3e6f62e1a715dbf87d55ac999 I0311 06:24:36.128854 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 10.352897ms to process 1 records I0311 06:24:36.129909 1 tasks_processing.go:74] worker 16 stopped. I0311 06:24:36.130118 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=77f3b92bd7b587f3afdca1880f6445f7941579d49ba65936dc20d1d1f565a8e2 I0311 06:24:36.130203 1 recorder.go:70] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=992838a8176b95d0696350537881569b55b86994e2784c85a77abfb1b81b4424 I0311 06:24:36.130216 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 31.622111ms to process 2 records I0311 06:24:36.130758 1 tasks_processing.go:74] worker 0 stopped. I0311 06:24:36.131359 1 recorder.go:70] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=40d31add9fd505de6644d7e517a045172cc7c37a333c544606f0373635e409c1 I0311 06:24:36.131383 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 13.610673ms to process 1 records I0311 06:24:36.133621 1 tasks_processing.go:74] worker 12 stopped. I0311 06:24:36.133642 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 8.784815ms to process 0 records I0311 06:24:36.133709 1 tasks_processing.go:74] worker 30 stopped. I0311 06:24:36.135514 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0311 06:24:36.135525 1 recorder.go:70] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-fcgvr with fingerprint=0732ced52b2b4e5a9dde1d33a83d995db0a36f967410c895ee86cc4dcc76b421 I0311 06:24:36.135618 1 recorder.go:70] Recording config/running_containers with fingerprint=0a0b7f85ae3a15cb70888a24807a35ca8be6120a77dad6024bcf781adc3cd79f I0311 06:24:36.135633 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 36.81524ms to process 2 records I0311 06:24:36.135643 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 12.999594ms to process 0 records E0311 06:24:36.135651 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0311 06:24:36.135655 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0311 06:24:36.135660 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 10.911916ms to process 0 records I0311 06:24:36.135667 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 14.539513ms to process 0 records I0311 06:24:36.135675 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 10.719159ms to process 0 records I0311 06:24:36.135682 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 13.39ms to process 0 records I0311 06:24:36.135691 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 14.932532ms to process 0 records I0311 06:24:36.135694 1 tasks_processing.go:74] worker 25 stopped. I0311 06:24:36.135697 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 15.389315ms to process 0 records I0311 06:24:36.135703 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0311 06:24:36.135710 1 tasks_processing.go:74] worker 29 stopped. I0311 06:24:36.135711 1 tasks_processing.go:74] worker 31 stopped. I0311 06:24:36.135715 1 tasks_processing.go:74] worker 7 stopped. I0311 06:24:36.135703 1 tasks_processing.go:74] worker 3 stopped. I0311 06:24:36.135718 1 tasks_processing.go:74] worker 18 stopped. I0311 06:24:36.135723 1 tasks_processing.go:74] worker 28 stopped. W0311 06:24:36.135681 1 operator.go:287] started I0311 06:24:36.138346 1 tasks_processing.go:74] worker 21 stopped. I0311 06:24:36.138432 1 recorder.go:70] Recording config/image with fingerprint=f61cb9d5bd5e4f1fdd3182f513441c5113412e8c94ab39cfb07e24bee1e43b39 I0311 06:24:36.138443 1 gather.go:177] gatherer "clusterconfig" function "image" took 13.420669ms to process 1 records I0311 06:24:36.142874 1 tasks_processing.go:74] worker 8 stopped. E0311 06:24:36.142885 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0311 06:24:36.142908 1 recorder.go:70] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0311 06:24:36.142916 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 18.787428ms to process 1 records I0311 06:24:36.146142 1 tasks_processing.go:74] worker 19 stopped. I0311 06:24:36.146287 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=33af8e320d8bad6b511dac017c97b4dc384aedd84cdaa3f8b2c048b159671a05 I0311 06:24:36.146352 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=0471350f097aeaef5f315180edcb2e5863f931e0a8ef79e20357a508e8cdd418 I0311 06:24:36.146405 1 recorder.go:70] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=94597b6c635d3997e6ca05c633de4801f0ad5363b4001bb40e747e166d8841f2 I0311 06:24:36.146419 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 22.082137ms to process 3 records I0311 06:24:36.146486 1 recorder.go:70] Recording config/proxy with fingerprint=100a51ed1526abd7916f5bf9679ae74add439c44c1d7f4373f4b42a6818feda5 I0311 06:24:36.146498 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 22.25023ms to process 1 records I0311 06:24:36.146503 1 tasks_processing.go:74] worker 24 stopped. I0311 06:24:36.146575 1 tasks_processing.go:74] worker 15 stopped. I0311 06:24:36.147972 1 recorder.go:70] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=32beaa8261403bdd4f6666d4bbbcd5a726cc21fcdf2c8afd5fbdd7f2897ac5f3 I0311 06:24:36.148518 1 recorder.go:70] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=7275a5c2f63e4eca29e7d8ac9d17d15cf96efd6c439be4e9304f0a6e821371ed I0311 06:24:36.148586 1 gather.go:177] gatherer "clusterconfig" function "crds" took 48.175045ms to process 2 records I0311 06:24:36.148724 1 recorder.go:70] Recording config/olm_operators with fingerprint=677275af4c6629cc0c8b567faf6df46ed838dc81c49e84913ca1f5e3493c7bae I0311 06:24:36.148755 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 49.565507ms to process 1 records I0311 06:24:36.148799 1 tasks_processing.go:74] worker 1 stopped. I0311 06:24:36.150807 1 prometheus_rules.go:88] Prometheus rules successfully created I0311 06:24:36.167489 1 controller.go:203] Source scaController *sca.Controller is not ready I0311 06:24:36.167504 1 controller.go:203] Source clusterTransferController *clustertransfer.Controller is not ready I0311 06:24:36.167509 1 controller.go:203] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0311 06:24:36.167512 1 controller.go:203] Source periodic-conditional *controllerstatus.Simple is not ready I0311 06:24:36.167516 1 controller.go:203] Source periodic-workloads *controllerstatus.Simple is not ready I0311 06:24:36.167533 1 controller.go:458] The operator is still being initialized I0311 06:24:36.167538 1 controller.go:481] The operator is healthy I0311 06:24:36.167722 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0311 06:24:36.167740 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0311 06:24:36.167745 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0311 06:24:36.172325 1 base_controller.go:82] Caches are synced for ConfigController I0311 06:24:36.172339 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... E0311 06:24:36.175990 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2728857ac8-334b-4db7-bfb8-855a066d7be5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:37916->172.30.0.10:53: read: connection refused I0311 06:24:36.176004 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2728857ac8-334b-4db7-bfb8-855a066d7be5%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:37916->172.30.0.10:53: read: connection refused I0311 06:24:36.192072 1 tasks_processing.go:74] worker 23 stopped. I0311 06:24:36.192095 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 67.383181ms to process 0 records I0311 06:24:36.192896 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 06:24:36.196082 1 tasks_processing.go:74] worker 22 stopped. E0311 06:24:36.196096 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0311 06:24:36.196103 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0311 06:24:36.196107 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0311 06:24:36.196118 1 recorder.go:70] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=295a2bc7810c628251f1f312c209b5eb4da888326706c635ac0c12aec0969e17 I0311 06:24:36.196144 1 recorder.go:70] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0311 06:24:36.196152 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0311 06:24:36.196157 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=fe7abfa7f7aea852d8bca6b7df9b4d5de32045254ff2774f9bd30f0b6dcb7dc4 I0311 06:24:36.196162 1 recorder.go:70] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0311 06:24:36.196202 1 recorder.go:70] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0311 06:24:36.196210 1 recorder.go:70] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0311 06:24:36.196216 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 98.151818ms to process 7 records I0311 06:24:36.211290 1 tasks_processing.go:74] worker 14 stopped. I0311 06:24:36.211955 1 recorder.go:70] Recording config/version with fingerprint=05046e1761fde757a4ebefe713d1d974fe8f2c0015783a1e863b768757771fc3 I0311 06:24:36.211973 1 recorder.go:70] Recording config/id with fingerprint=cc3aefef28aeaf75e314865484a660f4fb8020688f2b3e63046e229759800fc7 I0311 06:24:36.211981 1 gather.go:177] gatherer "clusterconfig" function "version" took 114.730101ms to process 2 records I0311 06:24:36.218385 1 tasks_processing.go:74] worker 13 stopped. I0311 06:24:36.219284 1 recorder.go:70] Recording config/node/ip-10-0-0-117.ec2.internal with fingerprint=0e2ab48fc3adf20465d139c3c1b4b5344df8936a2ddc702d63beddded32e0afb I0311 06:24:36.219465 1 recorder.go:70] Recording config/node/ip-10-0-1-123.ec2.internal with fingerprint=d35fe880d6bedcca701ac22c3c1df97b3f4df0e5b12d1d8c336c1185de2ab779 I0311 06:24:36.219637 1 recorder.go:70] Recording config/node/ip-10-0-2-178.ec2.internal with fingerprint=5fa7f71acb8b2b3143443827e4597963d30110d206827c039225cc48a09bb14a I0311 06:24:36.219678 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 93.35218ms to process 3 records I0311 06:24:36.229822 1 tasks_processing.go:74] worker 5 stopped. I0311 06:24:36.229839 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 106.955976ms to process 0 records I0311 06:24:36.235958 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0311 06:24:36.236026 1 base_controller.go:82] Caches are synced for LoggingSyncer I0311 06:24:36.236041 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0311 06:24:36.238714 1 tasks_processing.go:74] worker 10 stopped. E0311 06:24:36.238727 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0311 06:24:36.238732 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ovbeu6i8bpkjb7e0df1muj9ermnd94m-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ovbeu6i8bpkjb7e0df1muj9ermnd94m-primary-cert-bundle-secret" not found I0311 06:24:36.238895 1 recorder.go:70] Recording aggregated/ingress_controllers_certs with fingerprint=bd15c54cf369d82a967b047e234b3915afc520722dd7021ba60ec0bcd9b338e4 I0311 06:24:36.238908 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 141.938924ms to process 1 records W0311 06:24:36.239525 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:38761->172.30.0.10:53: read: connection refused I0311 06:24:36.239539 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:38761->172.30.0.10:53: read: connection refused I0311 06:24:36.281502 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress-operator" I0311 06:24:36.284542 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "dnsrecords" in API group "ingress.operator.openshift.io" in the namespace "openshift-ingress" I0311 06:24:36.530303 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0311 06:24:36.530399 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0311 06:24:36.530478 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-g5htd pod in namespace openshift-dns (previous: false). I0311 06:24:36.760251 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-g5htd pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-g5htd\" is waiting to start: ContainerCreating" I0311 06:24:36.760272 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-g5htd\" is waiting to start: ContainerCreating" I0311 06:24:36.760280 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-g5htd pod in namespace openshift-dns (previous: false). I0311 06:24:36.826401 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "ovn" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" I0311 06:24:36.933792 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-g5htd pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-g5htd\" is waiting to start: ContainerCreating" I0311 06:24:36.933815 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-g5htd\" is waiting to start: ContainerCreating" I0311 06:24:36.933846 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-j75x4 pod in namespace openshift-dns (previous: false). I0311 06:24:37.026435 1 gather_cluster_operators.go:184] Unable to get operatorpkis.network.operator.openshift.io resource due to: operatorpkis.network.operator.openshift.io "signer" is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot get resource "operatorpkis" in API group "network.operator.openshift.io" in the namespace "openshift-ovn-kubernetes" W0311 06:24:37.117687 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 06:24:37.158550 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-j75x4 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-j75x4\" is waiting to start: ContainerCreating" I0311 06:24:37.158585 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-j75x4\" is waiting to start: ContainerCreating" I0311 06:24:37.158594 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-j75x4 pod in namespace openshift-dns (previous: false). I0311 06:24:37.335337 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-j75x4 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-j75x4\" is waiting to start: ContainerCreating" I0311 06:24:37.335360 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-j75x4\" is waiting to start: ContainerCreating" I0311 06:24:37.335396 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-nzc8q pod in namespace openshift-dns (previous: false). I0311 06:24:37.560096 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-nzc8q pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-nzc8q\" is waiting to start: ContainerCreating" I0311 06:24:37.560115 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-nzc8q\" is waiting to start: ContainerCreating" I0311 06:24:37.560124 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-nzc8q pod in namespace openshift-dns (previous: false). I0311 06:24:37.734608 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-nzc8q pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-nzc8q\" is waiting to start: ContainerCreating" I0311 06:24:37.734639 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-nzc8q\" is waiting to start: ContainerCreating" I0311 06:24:37.734656 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-kg27l pod in namespace openshift-dns (previous: false). I0311 06:24:37.828139 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0311 06:24:37.934742 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 06:24:37.934776 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-n5k86 pod in namespace openshift-dns (previous: false). W0311 06:24:38.118252 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 06:24:38.134096 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 06:24:38.134112 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-rh9vq pod in namespace openshift-dns (previous: false). I0311 06:24:38.229752 1 tasks_processing.go:74] worker 11 stopped. I0311 06:24:38.229814 1 recorder.go:70] Recording config/clusteroperator/console with fingerprint=6d1bdf9de2541030b55ffabfc998167b0c5d891f80949249105b5212a93ff293 I0311 06:24:38.229856 1 recorder.go:70] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=46f1689ad7969e7bf328b69f7c0a3e0ecb8e7484ba3065367af1662e43c1014a I0311 06:24:38.229891 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0311 06:24:38.229923 1 recorder.go:70] Recording config/clusteroperator/dns with fingerprint=4d4e11b1eeb48eea3926959a48ec985eda42f3de56dfa236d333ba448283aad7 I0311 06:24:38.229944 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0311 06:24:38.229970 1 recorder.go:70] Recording config/clusteroperator/image-registry with fingerprint=7a314afa8fb73a53efb6583cf9b64a7ffc110f17ccbb22d392cad27845d87f76 I0311 06:24:38.230003 1 recorder.go:70] Recording config/clusteroperator/ingress with fingerprint=0e2f404678b9f6e02e3a9afd756230f1fb7be7ee4dcc5955fb6153bbd6cdaf6d I0311 06:24:38.230030 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=891c4b707b80fee6eec0e75131c3b5b40dbc55f809045a6164593190e44a3646 I0311 06:24:38.230060 1 recorder.go:70] Recording config/clusteroperator/insights with fingerprint=37fed25a3e4ec3e92b802fe8188dbed0c3743b9d98793dc60ad878f4c5d8a3f1 I0311 06:24:38.230071 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e5ff11d57817f84a678f6fa9565af55bd1120227c16a21933637ab62675a6d70 I0311 06:24:38.230088 1 recorder.go:70] Recording config/clusteroperator/kube-apiserver with fingerprint=5b081e320052562142c3b4c46b7236e52289fdb9ac25ad149726b1470a52015f I0311 06:24:38.230099 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0311 06:24:38.230116 1 recorder.go:70] Recording config/clusteroperator/kube-controller-manager with fingerprint=e435ecf0a58ba42c0c34ffc946fc34580a66823fd7651e003dd0d6ffba4d44b3 I0311 06:24:38.230126 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0311 06:24:38.230143 1 recorder.go:70] Recording config/clusteroperator/kube-scheduler with fingerprint=2fd3bd2b2efd57a30dab0404151513f9ee3828f3d1d0043ec3fefa95855de8ab I0311 06:24:38.230153 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0311 06:24:38.230173 1 recorder.go:70] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=a8287cc00dfa1de3ce5a5cca0ce91b0281fddeeda6e43105531aa02cfb7ed414 I0311 06:24:38.230184 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0311 06:24:38.230201 1 recorder.go:70] Recording config/clusteroperator/monitoring with fingerprint=8e37c64c1507f35195becec9a631b575f72794165e10059fcc4ccb16ea2a880e I0311 06:24:38.230321 1 recorder.go:70] Recording config/clusteroperator/network with fingerprint=f96e794e2c1daf22b2cd9e3afa22050d5c4038a1c321dfe83c814ae8705a5bec I0311 06:24:38.230346 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0311 06:24:38.230369 1 recorder.go:70] Recording config/clusteroperator/node-tuning with fingerprint=3e4950a0d97d3ae86d9cce926b211ec2391bdf368aa7badb63870454cd307e73 I0311 06:24:38.230394 1 recorder.go:70] Recording config/clusteroperator/openshift-apiserver with fingerprint=0dea2805e294b3c3dc9053b298910ec08a637d776bea1d469516f9e2ef8cf8a3 I0311 06:24:38.230404 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0311 06:24:38.230420 1 recorder.go:70] Recording config/clusteroperator/openshift-controller-manager with fingerprint=f58e2c14592f28f86ea0beb92d477fb440e75d1fc2815b116415a53c2bf1d4df I0311 06:24:38.230430 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0311 06:24:38.230444 1 recorder.go:70] Recording config/clusteroperator/openshift-samples with fingerprint=37c8afed20308f004843356efa1fc52f893f585dd5250361521f2a7c242c5a2e I0311 06:24:38.230459 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=76f0f3049d583cbe139d7334b48149a8e1171eb040be731fbafa40add77109a3 I0311 06:24:38.230475 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=7431df644ae30901decb13057129943fb7bcddabb0321de61a0929c952e492ca I0311 06:24:38.230494 1 recorder.go:70] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=ee11c9b74465c53ed166af3e89b9446f4c405b029928922f0efcab857b768b20 I0311 06:24:38.230515 1 recorder.go:70] Recording config/clusteroperator/service-ca with fingerprint=3c9812a653f81ed2f5f8b14e709832b994bc8c572904453da84ba6b60d3ed5e6 I0311 06:24:38.230547 1 recorder.go:70] Recording config/clusteroperator/storage with fingerprint=40013c0952ea542f9c348ef7fc39495e97e833b1c2aac06cc6e8347e3b910234 I0311 06:24:38.230586 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0311 06:24:38.230599 1 recorder.go:70] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0311 06:24:38.230606 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.107779816s to process 34 records I0311 06:24:38.334430 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 06:24:38.334515 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6bdb6495df-dn7vd pod in namespace openshift-image-registry (previous: false). I0311 06:24:38.534155 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6bdb6495df-dn7vd pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6bdb6495df-dn7vd\" is waiting to start: ContainerCreating" I0311 06:24:38.534174 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6bdb6495df-dn7vd\" is waiting to start: ContainerCreating" I0311 06:24:38.534219 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6bdb6495df-zdbn2 pod in namespace openshift-image-registry (previous: false). I0311 06:24:38.734862 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6bdb6495df-zdbn2 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6bdb6495df-zdbn2\" is waiting to start: ContainerCreating" I0311 06:24:38.734881 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6bdb6495df-zdbn2\" is waiting to start: ContainerCreating" I0311 06:24:38.734928 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7dcc7b8754-9gc9q pod in namespace openshift-image-registry (previous: false). I0311 06:24:38.934488 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7dcc7b8754-9gc9q pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7dcc7b8754-9gc9q\" is waiting to start: ContainerCreating" I0311 06:24:38.934504 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7dcc7b8754-9gc9q\" is waiting to start: ContainerCreating" I0311 06:24:38.934515 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9kpg9 pod in namespace openshift-image-registry (previous: false). W0311 06:24:39.118378 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 06:24:39.134946 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 06:24:39.134963 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-bl99l pod in namespace openshift-image-registry (previous: false). I0311 06:24:39.335135 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 06:24:39.335155 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-fqfdc pod in namespace openshift-image-registry (previous: false). I0311 06:24:39.537271 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0311 06:24:39.537292 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-58cf7f4d5d-dhhdc pod in namespace openshift-ingress (previous: false). I0311 06:24:39.734230 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-58cf7f4d5d-dhhdc pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-58cf7f4d5d-dhhdc\" is waiting to start: ContainerCreating" I0311 06:24:39.734248 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-58cf7f4d5d-dhhdc\" is waiting to start: ContainerCreating" I0311 06:24:39.734258 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-58cf7f4d5d-kmpch pod in namespace openshift-ingress (previous: false). I0311 06:24:39.935776 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-58cf7f4d5d-kmpch pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-58cf7f4d5d-kmpch\" is waiting to start: ContainerCreating" I0311 06:24:39.935802 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-58cf7f4d5d-kmpch\" is waiting to start: ContainerCreating" I0311 06:24:39.935812 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-69fc795457-zgdpv pod in namespace openshift-ingress (previous: false). W0311 06:24:40.118274 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0311 06:24:40.134648 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-69fc795457-zgdpv pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-69fc795457-zgdpv\" is waiting to start: ContainerCreating" I0311 06:24:40.134666 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-69fc795457-zgdpv\" is waiting to start: ContainerCreating" I0311 06:24:40.134695 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dbpj7 pod in namespace openshift-ingress-canary (previous: false). I0311 06:24:40.334861 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dbpj7 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dbpj7\" is waiting to start: ContainerCreating" I0311 06:24:40.334881 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dbpj7\" is waiting to start: ContainerCreating" I0311 06:24:40.334914 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-fq8pk pod in namespace openshift-ingress-canary (previous: false). I0311 06:24:40.534771 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-fq8pk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fq8pk\" is waiting to start: ContainerCreating" I0311 06:24:40.534789 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fq8pk\" is waiting to start: ContainerCreating" I0311 06:24:40.534819 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-xtbpg pod in namespace openshift-ingress-canary (previous: false). I0311 06:24:40.733936 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-xtbpg pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-xtbpg\" is waiting to start: ContainerCreating" I0311 06:24:40.733953 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-xtbpg\" is waiting to start: ContainerCreating" I0311 06:24:40.733994 1 tasks_processing.go:74] worker 6 stopped. I0311 06:24:40.734081 1 recorder.go:70] Recording events/openshift-dns-operator with fingerprint=7bf3a2b9a39533311eddd283130ac6ca5ba82a7380762f54c097a90adae377df I0311 06:24:40.734132 1 recorder.go:70] Recording events/openshift-dns with fingerprint=376754d8fa082bcdb4a8e324c5c764e62fb2eed0258fb0fbd325a91d17c7d6f0 I0311 06:24:40.734228 1 recorder.go:70] Recording events/openshift-image-registry with fingerprint=644363d52b7464ce9c40c3c44057b4e7646c2f24238473713ee05e631a9f1ce8 I0311 06:24:40.734260 1 recorder.go:70] Recording events/openshift-ingress-operator with fingerprint=3982238ec8427eadb1a7adb5db25ba3fb115d423ae9856d7f92423161cd81c86 I0311 06:24:40.734312 1 recorder.go:70] Recording events/openshift-ingress with fingerprint=053fc47dfcfbbe9e9da1c2156b504f60876cc52efefd4c2c773941a48fd139a7 I0311 06:24:40.734330 1 recorder.go:70] Recording events/openshift-ingress-canary with fingerprint=b5f5676622a6875a0e5ce83f4e39a63d6dbbd7c0236ae4733357b182bfa7b6ff I0311 06:24:40.734473 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-g5htd with fingerprint=4a13f4d671e3cdb5c1e72b2f4647a158386b579dcf5540ffce4f580c9d52d02b I0311 06:24:40.734646 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-j75x4 with fingerprint=0f348e3a7e511ba4ee3d715296851b4752f32f330006a72beef53ebd81e327bc I0311 06:24:40.734737 1 recorder.go:70] Recording config/pod/openshift-dns/dns-default-nzc8q with fingerprint=f85f409e936838739174ebeba6a02f25da293c66eb4d4062db32d0157f3f8102 I0311 06:24:40.734855 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-6bdb6495df-dn7vd with fingerprint=82689dad2fb6efa754f94f5e83642628457c24313581570a753122e35148954e I0311 06:24:40.734959 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-6bdb6495df-zdbn2 with fingerprint=4125f3c8fe225367180d0451d2b966d1c15888834f7a7723477f48b789fdcf96 I0311 06:24:40.735071 1 recorder.go:70] Recording config/pod/openshift-image-registry/image-registry-7dcc7b8754-9gc9q with fingerprint=6fd0d8f23177338119f1d96e26a5861c4675d7db761dd032f86121926324e108 I0311 06:24:40.735140 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-dbpj7 with fingerprint=33efa22e161d6997016b6a7a33443f0b26c8d6bcb9728c4a6811d4d9e12bf5ae I0311 06:24:40.735198 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-fq8pk with fingerprint=f5ace866104e38d1f6b36613f3953e625c0bcc0400f8c531251ea9da73377f0b I0311 06:24:40.735256 1 recorder.go:70] Recording config/pod/openshift-ingress-canary/ingress-canary-xtbpg with fingerprint=0e535d9d5d567bd86d48abbb8a9175c32021f7125bbbd5a446d312a6f2dd21ba I0311 06:24:40.735266 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.636121455s to process 15 records W0311 06:24:41.118459 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0311 06:24:41.118486 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0311 06:24:41.118498 1 tasks_processing.go:74] worker 26 stopped. E0311 06:24:41.118509 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0311 06:24:41.118522 1 recorder.go:70] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0311 06:24:41.118536 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0311 06:24:41.118546 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.021919364s to process 1 records I0311 06:24:48.368918 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 06:24:48.527145 1 tasks_processing.go:74] worker 9 stopped. I0311 06:24:48.527200 1 recorder.go:70] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0311 06:24:48.527217 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.415508084s to process 1 records I0311 06:24:49.526166 1 tasks_processing.go:74] worker 4 stopped. I0311 06:24:49.526532 1 recorder.go:70] Recording config/serviceaccounts with fingerprint=bf87f2531cc0884b07268060c8c965a23390b0089a10c3b066309a34832b594a I0311 06:24:49.526575 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.40635433s to process 1 records E0311 06:24:49.526660 1 periodic.go:250] "Unhandled Error" err="clusterconfig failed after 13.43s with: function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0311 06:24:49.527775 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0311 06:24:49.527795 1 periodic.go:212] Running workloads gatherer I0311 06:24:49.527814 1 tasks_processing.go:45] number of workers: 2 I0311 06:24:49.527829 1 tasks_processing.go:69] worker 1 listening for tasks. I0311 06:24:49.527834 1 tasks_processing.go:71] worker 1 working on workload_info task. I0311 06:24:49.527904 1 tasks_processing.go:69] worker 0 listening for tasks. I0311 06:24:49.527946 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0311 06:24:49.552925 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0311 06:24:49.555253 1 tasks_processing.go:74] worker 0 stopped. I0311 06:24:49.555277 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 27.284699ms to process 0 records I0311 06:24:49.560860 1 gather_workloads_info.go:387] No image sha256:a7c71f3c9714cf1717c45e12ec817be6d6209f00989e6a31d634e527f0c8147d (9ms) I0311 06:24:49.567962 1 gather_workloads_info.go:387] No image sha256:bae7a33f8db8a3d4b3c4c05498aba85a0ce463f85322067c48e79663710e616e (7ms) I0311 06:24:49.574933 1 gather_workloads_info.go:387] No image sha256:18956d5fbc8d0369a26cb1e00eb5c3aff1973ae8cbd3983920a0cf2851f05358 (7ms) I0311 06:24:49.581491 1 gather_workloads_info.go:387] No image sha256:0a8a0473029ace3adbb66f490dbe560f0e7782a38cdf32c1f7dd3e092e1d191e (7ms) I0311 06:24:49.588531 1 gather_workloads_info.go:387] No image sha256:9697bc2258bdfa9ae8c1866cc7eb0b3b46851998db827b121bdf77417a881eb3 (7ms) I0311 06:24:49.595605 1 gather_workloads_info.go:387] No image sha256:7941b5d9a758b8667e99cd9236f7f96ec036af61df5cbc96e16077d36700d7c7 (7ms) I0311 06:24:49.602509 1 gather_workloads_info.go:387] No image sha256:824a3b8f78e19aa21a9f6444aefe8e0b624886cca18fe828b13acbab55e6e868 (7ms) I0311 06:24:49.609478 1 gather_workloads_info.go:387] No image sha256:ceab8a8c340801fcebf800620d3fef5493d8a443077a21c6e56984051bd3abde (7ms) I0311 06:24:49.616347 1 gather_workloads_info.go:387] No image sha256:752be69b2262be713df12c47f4bac8c2dafed272c401e9a89f8060f053d68054 (7ms) I0311 06:24:49.624611 1 gather_workloads_info.go:387] No image sha256:2fac754deaeade3456361eed52e344318ff16d04819384432759f0ea35530114 (8ms) I0311 06:24:49.661501 1 gather_workloads_info.go:387] No image sha256:f2c46054a8f64a0e949cbf30295f31b4a35a0203c6ad03fa7ec922b4101dcbf8 (37ms) I0311 06:24:49.761833 1 gather_workloads_info.go:387] No image sha256:3d0634fe58641d5242649c44ddba70ed67fdd6d2dcb5c2261df5cee8b33de9fd (100ms) I0311 06:24:49.862047 1 gather_workloads_info.go:387] No image sha256:a9e885d6f0456a2cc10f9e5da71fe9403f5e4c639b9b7ea15bd403b272ccb824 (100ms) I0311 06:24:49.963619 1 gather_workloads_info.go:387] No image sha256:8ef4bdf07dda423fab73484dbc66d527ce8ba8cc2d4f99210bc1bc24ea08c0cb (102ms) I0311 06:24:50.061147 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (98ms) I0311 06:24:50.162354 1 gather_workloads_info.go:387] No image sha256:373614619b9420b110d1508c5f17e066ce69c4c226fd04d02fbb959d9ba41eb6 (101ms) I0311 06:24:50.261743 1 gather_workloads_info.go:387] No image sha256:ccdda82b4adfa8ee4a84b3bad19693be6a42a9881a7b83788d835912129c49a4 (99ms) I0311 06:24:50.361419 1 gather_workloads_info.go:387] No image sha256:0a92dd43975f972e3dc707fb37854e046813531553038147e9c90d54a8d9df73 (100ms) I0311 06:24:50.462415 1 gather_workloads_info.go:387] No image sha256:6e67c980b7300e769fb4b2adaaf006d0f8274e43b10586701b204ea5153f15fc (101ms) I0311 06:24:50.561374 1 gather_workloads_info.go:387] No image sha256:8b1e32ce43eb1247b0558230b6e0baa85ac62f1a5e2a2089d40b4fea90538529 (99ms) I0311 06:24:50.661623 1 gather_workloads_info.go:387] No image sha256:420c12f61c53e54eab2d99476759c0de339fca98ce0a3a782bc6545cc0e97a9c (100ms) I0311 06:24:50.761601 1 gather_workloads_info.go:387] No image sha256:87efe06afd1f04426fca7f86c0f74c4ee75c311ba199ddabd5b849b877bc59fa (100ms) I0311 06:24:50.863648 1 gather_workloads_info.go:387] No image sha256:6d6ebff54b8adac74f4d1b12ac8aa0f16cd7b28370e0d6aa847d1c457e03a5b6 (102ms) I0311 06:24:50.966016 1 gather_workloads_info.go:387] No image sha256:d28734effaadd66434e77a1a5fbe2e8a4ca2066cd9f8868c22ade9475539bfd7 (102ms) I0311 06:24:51.055623 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 06:24:51.060820 1 gather_workloads_info.go:387] No image sha256:2d077e73dd76873ee2a1583aa3b3e76a1b408737f0af2e22b2aa055604d89e81 (95ms) I0311 06:24:51.160853 1 gather_workloads_info.go:387] No image sha256:4f175ee49f51dc4379e9993fecd1657c7a9c4c3dc096772cb198d0212b9eea47 (100ms) I0311 06:24:51.255833 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 06:24:51.260836 1 gather_workloads_info.go:387] No image sha256:944d9261ba7a143131fe8267c172defc5f37acc1cea3d4d373ec6fc5d8bfcc31 (100ms) I0311 06:24:51.361036 1 gather_workloads_info.go:387] No image sha256:23c4fe84047a4e6cbe7f75f470e7d6fe0e61e7910dd17945b1b61bc4b72f3f2a (100ms) I0311 06:24:51.460724 1 gather_workloads_info.go:387] No image sha256:76f820bd9bd138d29305d545d7d49bfe63e2923f1fb1ec2e8eac81a388359024 (100ms) I0311 06:24:51.561144 1 gather_workloads_info.go:387] No image sha256:48404bd61d05dc738ccce4d22e36b30dbcdc6015b06b4afb604fd1baeee35bf2 (100ms) I0311 06:24:51.662834 1 gather_workloads_info.go:387] No image sha256:14703f73bb1ca69ea03b726e340b3b68f8e294e1f22c80d1a16666ea3d4a88a3 (102ms) I0311 06:24:51.662869 1 tasks_processing.go:74] worker 1 stopped. E0311 06:24:51.662885 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0311 06:24:51.663181 1 recorder.go:70] Recording config/workload_info with fingerprint=c36826ff6784bfcd4e54316959421ce1b9cc001f9f0d2228fce1a39e3ff773ea I0311 06:24:51.663205 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.135027342s to process 1 records E0311 06:24:51.663237 1 periodic.go:250] "Unhandled Error" err="workloads failed after 2.135s with: function \"workload_info\" failed with an error" I0311 06:24:51.664344 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0311 06:24:51.664361 1 periodic.go:212] Running conditional gatherer I0311 06:24:51.672272 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules I0311 06:24:51.685864 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.10:50760->172.30.0.10:53: read: connection refused E0311 06:24:51.686284 1 conditional_gatherer.go:320] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0311 06:24:51.686362 1 conditional_gatherer.go:382] updating version cache for conditional gatherer I0311 06:24:51.694605 1 conditional_gatherer.go:390] cluster version is '4.19.9' E0311 06:24:51.694621 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694626 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694630 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694633 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694636 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694640 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694644 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694647 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing E0311 06:24:51.694650 1 conditional_gatherer.go:207] error checking conditions for a gathering rule: alerts cache is missing I0311 06:24:51.694665 1 tasks_processing.go:45] number of workers: 3 I0311 06:24:51.694672 1 tasks_processing.go:69] worker 2 listening for tasks. I0311 06:24:51.694677 1 tasks_processing.go:71] worker 2 working on remote_configuration task. I0311 06:24:51.694682 1 tasks_processing.go:69] worker 0 listening for tasks. I0311 06:24:51.694693 1 tasks_processing.go:69] worker 1 listening for tasks. I0311 06:24:51.694706 1 tasks_processing.go:74] worker 1 stopped. I0311 06:24:51.694717 1 tasks_processing.go:71] worker 0 working on rapid_container_logs task. I0311 06:24:51.694719 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0311 06:24:51.694746 1 recorder.go:70] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0311 06:24:51.694766 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 989ns to process 1 records I0311 06:24:51.694834 1 recorder.go:70] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0311 06:24:51.694846 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.494µs to process 1 records I0311 06:24:51.694847 1 tasks_processing.go:74] worker 2 stopped. I0311 06:24:51.695004 1 tasks_processing.go:74] worker 0 stopped. I0311 06:24:51.695015 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 273.003µs to process 0 records I0311 06:24:51.695046 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.19.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.10:50760->172.30.0.10:53: read: connection refused I0311 06:24:51.695065 1 recorder.go:70] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0311 06:24:51.716290 1 recorder.go:70] Recording insights-operator/gathers with fingerprint=410179d49dc69ace2c4bbff226148115b7fd37a0ade36e0c2d05a21df907032d I0311 06:24:51.716433 1 diskrecorder.go:70] Writing 108 records to /var/lib/insights-operator/insights-2026-03-11-062451.tar.gz I0311 06:24:51.723758 1 diskrecorder.go:51] Wrote 108 records to disk in 7ms I0311 06:24:51.723793 1 periodic.go:281] Gathering cluster info every 2h0m0s I0311 06:24:51.723809 1 periodic.go:282] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0311 06:25:04.187656 1 configmapobserver.go:84] configmaps "insights-config" not found I0311 06:25:55.892816 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="da5aef361f6d5d1c4ab55571ab23e8076b169f420cecd0c76ae1027138fdab55") W0311 06:25:55.892848 1 builder.go:160] Restart triggered because of file /var/run/configmaps/service-ca-bundle/service-ca.crt was created I0311 06:25:55.892920 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0311 06:25:55.892927 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="8d83319af470cf3d535c05a73bfe66542a6e9de6594e453dd36cdb7917a89ce6") I0311 06:25:55.892939 1 base_controller.go:181] Shutting down ConfigController ... I0311 06:25:55.892982 1 periodic.go:173] Shutting down I0311 06:25:55.892994 1 base_controller.go:181] Shutting down LoggingSyncer ... I0311 06:25:55.893005 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0311 06:25:55.893007 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="e02b9fedc6db243461b2a85cc68e7258f28acf4273479ec4d8d2febc98adf1f5")