I0703 14:21:08.549135 1 cmd.go:253] Using service-serving-cert provided certificates I0703 14:21:08.550571 1 observer_polling.go:159] Starting file observer I0703 14:21:08.846827 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0703 14:21:08.847039 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0703 14:21:08.848112 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0703 14:21:08.848193 1 secure_serving.go:57] Forcing use of http/1.1 only W0703 14:21:08.848305 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0703 14:21:08.848346 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0703 14:21:08.848388 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0703 14:21:08.848419 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0703 14:21:08.848448 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0703 14:21:08.848474 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0703 14:21:08.852594 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0703 14:21:08.852661 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"d3bbf11a-b651-4152-8b98-8ab4bb87b8cb", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0703 14:21:08.853925 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0703 14:21:08.853930 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0703 14:21:08.853939 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0703 14:21:08.853946 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0703 14:21:08.853952 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0703 14:21:08.853944 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0703 14:21:08.854252 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0703 14:21:08.854581 1 secure_serving.go:213] Serving securely on [::]:8443 I0703 14:21:08.854609 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0703 14:21:08.857684 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0703 14:21:08.857708 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0703 14:21:08.857824 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0703 14:21:08.863926 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 14:21:08.863948 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0703 14:21:08.868824 1 secretconfigobserver.go:119] support secret does not exist I0703 14:21:08.873723 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 14:21:08.878342 1 secretconfigobserver.go:119] support secret does not exist I0703 14:21:08.881889 1 recorder.go:161] Pruning old reports every 4h15m42s, max age is 288h0m0s I0703 14:21:08.886951 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0703 14:21:08.886969 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0703 14:21:08.886991 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0703 14:21:08.887002 1 insightsreport.go:296] Starting report retriever I0703 14:21:08.887010 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0703 14:21:08.886990 1 periodic.go:209] Running clusterconfig gatherer I0703 14:21:08.887062 1 tasks_processing.go:45] number of workers: 32 I0703 14:21:08.887094 1 tasks_processing.go:69] worker 3 listening for tasks. I0703 14:21:08.887104 1 tasks_processing.go:71] worker 3 working on operators task. I0703 14:21:08.887107 1 tasks_processing.go:69] worker 31 listening for tasks. I0703 14:21:08.887108 1 tasks_processing.go:69] worker 1 listening for tasks. I0703 14:21:08.887115 1 tasks_processing.go:69] worker 18 listening for tasks. I0703 14:21:08.887122 1 tasks_processing.go:69] worker 19 listening for tasks. I0703 14:21:08.887125 1 tasks_processing.go:69] worker 17 listening for tasks. I0703 14:21:08.887129 1 tasks_processing.go:69] worker 0 listening for tasks. I0703 14:21:08.887132 1 tasks_processing.go:69] worker 20 listening for tasks. I0703 14:21:08.887124 1 tasks_processing.go:69] worker 10 listening for tasks. I0703 14:21:08.887134 1 tasks_processing.go:69] worker 7 listening for tasks. I0703 14:21:08.887140 1 tasks_processing.go:69] worker 21 listening for tasks. I0703 14:21:08.887138 1 tasks_processing.go:69] worker 11 listening for tasks. I0703 14:21:08.887142 1 tasks_processing.go:69] worker 8 listening for tasks. I0703 14:21:08.887145 1 tasks_processing.go:69] worker 6 listening for tasks. I0703 14:21:08.887149 1 tasks_processing.go:69] worker 9 listening for tasks. I0703 14:21:08.887147 1 tasks_processing.go:69] worker 12 listening for tasks. I0703 14:21:08.887154 1 tasks_processing.go:69] worker 4 listening for tasks. I0703 14:21:08.887148 1 tasks_processing.go:69] worker 22 listening for tasks. I0703 14:21:08.887159 1 tasks_processing.go:69] worker 2 listening for tasks. I0703 14:21:08.887168 1 tasks_processing.go:69] worker 24 listening for tasks. I0703 14:21:08.887168 1 tasks_processing.go:69] worker 5 listening for tasks. I0703 14:21:08.887173 1 tasks_processing.go:69] worker 27 listening for tasks. I0703 14:21:08.887168 1 tasks_processing.go:69] worker 14 listening for tasks. I0703 14:21:08.887176 1 tasks_processing.go:69] worker 25 listening for tasks. I0703 14:21:08.887155 1 tasks_processing.go:69] worker 13 listening for tasks. I0703 14:21:08.887183 1 tasks_processing.go:69] worker 29 listening for tasks. I0703 14:21:08.887183 1 tasks_processing.go:69] worker 26 listening for tasks. I0703 14:21:08.887185 1 tasks_processing.go:69] worker 30 listening for tasks. I0703 14:21:08.887155 1 tasks_processing.go:69] worker 23 listening for tasks. I0703 14:21:08.887175 1 tasks_processing.go:69] worker 15 listening for tasks. I0703 14:21:08.887194 1 tasks_processing.go:71] worker 31 working on cluster_apiserver task. I0703 14:21:08.887197 1 tasks_processing.go:71] worker 1 working on nodenetworkstates task. I0703 14:21:08.887200 1 tasks_processing.go:71] worker 15 working on silenced_alerts task. I0703 14:21:08.887203 1 tasks_processing.go:71] worker 19 working on sap_datahubs task. I0703 14:21:08.887240 1 tasks_processing.go:71] worker 4 working on qemu_kubevirt_launcher_logs task. I0703 14:21:08.887304 1 tasks_processing.go:71] worker 12 working on machine_config_pools task. I0703 14:21:08.887199 1 tasks_processing.go:71] worker 23 working on version task. I0703 14:21:08.887259 1 tasks_processing.go:71] worker 14 working on validating_webhook_configurations task. I0703 14:21:08.887184 1 tasks_processing.go:69] worker 16 listening for tasks. I0703 14:21:08.887770 1 tasks_processing.go:71] worker 16 working on jaegers task. I0703 14:21:08.887193 1 tasks_processing.go:71] worker 18 working on support_secret task. I0703 14:21:08.887266 1 tasks_processing.go:71] worker 22 working on oauths task. I0703 14:21:08.887272 1 tasks_processing.go:71] worker 2 working on certificate_signing_requests task. I0703 14:21:08.887267 1 tasks_processing.go:71] worker 21 working on pdbs task. I0703 14:21:08.887176 1 tasks_processing.go:69] worker 28 listening for tasks. I0703 14:21:08.887989 1 tasks_processing.go:71] worker 28 working on operators_pods_and_events task. I0703 14:21:08.887276 1 tasks_processing.go:71] worker 24 working on container_runtime_configs task. I0703 14:21:08.887277 1 tasks_processing.go:71] worker 17 working on openshift_machine_api_events task. I0703 14:21:08.887279 1 tasks_processing.go:71] worker 29 working on config_maps task. I0703 14:21:08.887282 1 tasks_processing.go:71] worker 0 working on schedulers task. I0703 14:21:08.887280 1 tasks_processing.go:71] worker 5 working on container_images task. I0703 14:21:08.887286 1 tasks_processing.go:71] worker 20 working on machine_autoscalers task. I0703 14:21:08.887284 1 tasks_processing.go:71] worker 27 working on image task. I0703 14:21:08.887284 1 tasks_processing.go:71] worker 25 working on dvo_metrics task. I0703 14:21:08.887287 1 tasks_processing.go:71] worker 13 working on proxies task. I0703 14:21:08.887290 1 tasks_processing.go:71] worker 10 working on machine_healthchecks task. I0703 14:21:08.887292 1 tasks_processing.go:71] worker 6 working on mutating_webhook_configurations task. I0703 14:21:08.887291 1 tasks_processing.go:71] worker 26 working on sap_config task. I0703 14:21:08.887293 1 tasks_processing.go:71] worker 7 working on openstack_dataplanedeployments task. I0703 14:21:08.887296 1 tasks_processing.go:71] worker 9 working on authentication task. I0703 14:21:08.887296 1 tasks_processing.go:71] worker 11 working on node_logs task. I0703 14:21:08.887297 1 tasks_processing.go:71] worker 30 working on machine_configs task. I0703 14:21:08.887298 1 tasks_processing.go:71] worker 8 working on monitoring_persistent_volumes task. I0703 14:21:08.892370 1 tasks_processing.go:71] worker 19 working on cost_management_metrics_configs task. I0703 14:21:08.892376 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 5.156491ms to process 0 records I0703 14:21:08.892390 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 5.165233ms to process 0 records I0703 14:21:08.892416 1 tasks_processing.go:71] worker 1 working on nodenetworkconfigurationpolicies task. E0703 14:21:08.893180 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:60852->172.30.0.10:53: read: connection refused I0703 14:21:08.893198 1 tasks_processing.go:71] worker 15 working on image_pruners task. E0703 14:21:08.893230 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:60852->172.30.0.10:53: read: connection refused I0703 14:21:08.893257 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 5.987966ms to process 0 records I0703 14:21:08.893671 1 tasks_processing.go:71] worker 31 working on storage_classes task. I0703 14:21:08.893850 1 recorder.go:75] Recording config/apiserver with fingerprint=4e7c2d8defe9234663ac114d244200270ca21ef5a6c59cc02f89d821e3d5e923 I0703 14:21:08.893865 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 6.46288ms to process 1 records I0703 14:21:08.894562 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0703 14:21:08.894582 1 controller.go:326] The initial operator extension status is healthy I0703 14:21:08.894594 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0703 14:21:08.894599 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0703 14:21:08.894603 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0703 14:21:08.894631 1 controller.go:489] The operator is still being initialized I0703 14:21:08.894643 1 controller.go:512] The operator is healthy I0703 14:21:08.894726 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0703 14:21:08.894832 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0703 14:21:08.894994 1 operator.go:288] started I0703 14:21:08.895017 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0703 14:21:08.905099 1 tasks_processing.go:71] worker 16 working on storage_cluster task. I0703 14:21:08.905109 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 17.312582ms to process 0 records I0703 14:21:08.905256 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 17.024094ms to process 0 records I0703 14:21:08.905266 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 16.633546ms to process 0 records E0703 14:21:08.905273 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0703 14:21:08.905277 1 tasks_processing.go:71] worker 24 working on crds task. I0703 14:21:08.905282 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 17.296779ms to process 0 records I0703 14:21:08.905291 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 17.943337ms to process 0 records I0703 14:21:08.905338 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 15.726863ms to process 0 records I0703 14:21:08.905377 1 gather_logs.go:145] no pods in namespace were found I0703 14:21:08.905397 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 18.084427ms to process 0 records I0703 14:21:08.905435 1 tasks_processing.go:71] worker 27 working on clusterroles task. I0703 14:21:08.905534 1 recorder.go:75] Recording config/image with fingerprint=01bbf63435b9680289078e719c60735ff28de60b46cfa70afdeb30df0a06cc7d I0703 14:21:08.905607 1 gather.go:177] gatherer "clusterconfig" function "image" took 16.846687ms to process 1 records I0703 14:21:08.905436 1 tasks_processing.go:71] worker 20 working on feature_gates task. I0703 14:21:08.905727 1 tasks_processing.go:71] worker 22 working on machines task. I0703 14:21:08.905441 1 tasks_processing.go:71] worker 12 working on openstack_dataplanenodesets task. I0703 14:21:08.905445 1 tasks_processing.go:71] worker 18 working on overlapping_namespace_uids task. I0703 14:21:08.905902 1 recorder.go:75] Recording config/oauth with fingerprint=9b89b37e9899248f99814250386cbd2c4383f64efd7d86a63c8008b6b7b6fe5e I0703 14:21:08.905913 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 17.669545ms to process 1 records I0703 14:21:08.905451 1 tasks_processing.go:71] worker 4 working on openshift_logging task. I0703 14:21:08.905453 1 tasks_processing.go:71] worker 7 working on machine_sets task. I0703 14:21:08.906367 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e543a6c3b35fbb062162a9b6b1a781d6b6a8c609faccfb3c98aacd5316b381e3 I0703 14:21:08.906453 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 17.363191ms to process 1 records I0703 14:21:08.906645 1 tasks_processing.go:71] worker 0 working on sap_pods task. I0703 14:21:08.909280 1 tasks_processing.go:71] worker 26 working on openstack_controlplanes task. I0703 14:21:08.909316 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 20.098602ms to process 0 records I0703 14:21:08.909542 1 recorder.go:75] Recording config/proxy with fingerprint=33a8ef86d3a6d54b60f5b1d00ddc4febe399615fcc2a364735ab2240128c232f I0703 14:21:08.909564 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 20.647552ms to process 1 records I0703 14:21:08.909572 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 17.061976ms to process 0 records I0703 14:21:08.909578 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 17.078849ms to process 0 records E0703 14:21:08.909586 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0703 14:21:08.909594 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 20.512193ms to process 0 records I0703 14:21:08.909607 1 tasks_processing.go:71] worker 10 working on networks task. I0703 14:21:08.909638 1 tasks_processing.go:71] worker 13 working on image_registries task. I0703 14:21:08.909674 1 tasks_processing.go:71] worker 19 working on aggregated_monitoring_cr_names task. I0703 14:21:08.909696 1 tasks_processing.go:71] worker 1 working on olm_operators task. I0703 14:21:08.910057 1 tasks_processing.go:71] worker 15 working on metrics task. I0703 14:21:08.910510 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=b9f1863709f89bbdbfee92d1f926fb4e5937923c9c8953c486d697d2b82a6a20 I0703 14:21:08.910586 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 16.849417ms to process 1 records I0703 14:21:08.910599 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 19.735756ms to process 0 records I0703 14:21:08.910607 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 22.092986ms to process 0 records I0703 14:21:08.910616 1 tasks_processing.go:71] worker 17 working on ingress_certificates task. I0703 14:21:08.910736 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0703 14:21:08.910754 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0703 14:21:08.910764 1 controller.go:212] Source scaController *sca.Controller is not ready I0703 14:21:08.910769 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0703 14:21:08.910775 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0703 14:21:08.910802 1 controller.go:489] The operator is still being initialized I0703 14:21:08.910853 1 controller.go:512] The operator is healthy I0703 14:21:08.910935 1 recorder.go:75] Recording config/authentication with fingerprint=bd3b7079eb05b0fd0842afe17c22615a7a253b8e559bc6f7adf8a3101c82c67c I0703 14:21:08.910956 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 21.019818ms to process 1 records I0703 14:21:08.911051 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=a2dcf54b17ef2392f632241f1d13eeb53ab3030d0d28e12ac8a5ac55e3a1512e I0703 14:21:08.911116 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=0927d515667188ff6904acb3ada2c472d23be404279beded2240b997cedc70ed I0703 14:21:08.911175 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=da6c1788d70d66887be8812e7cfaa55199fdb785e05a5e708de7c0ba94baf681 I0703 14:21:08.911211 1 tasks_processing.go:71] worker 21 working on ceph_cluster task. I0703 14:21:08.911233 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=c73f24333685ae2df02c4c0924fd289aab4fb0573e16f25a9c8448b0ac6b0f08 I0703 14:21:08.911379 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=fdf9cfbf8f8117bd10ceacd93cef1245058af5b1e02142e9dc9f3b4428a3b7bb I0703 14:21:08.911421 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=4d9d61cf0ca07ff1708e5e96f68583921f696553e4b60d8aef754fd148f4222b I0703 14:21:08.911464 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=d57f2270493248ea2d489bf480021338fdebab7110909ead8661a6164b3601a3 I0703 14:21:08.911504 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=85833ddb2309c7ac66c4bab03e284492de0e712e54080cff3b14a2bca95192c3 I0703 14:21:08.911560 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=be102deeccfa65500bbc8ff4f88102f49a89638ae68248666b3bcacea1e327f2 I0703 14:21:08.911243 1 tasks_processing.go:71] worker 9 working on pod_network_connectivity_checks task. I0703 14:21:08.911585 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=04f74d962845d1d0b0b3627588bf9fd52b574726a3669f51aabf454643500de9 I0703 14:21:08.911594 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 22.843801ms to process 10 records I0703 14:21:08.911252 1 tasks_processing.go:71] worker 8 working on ingress task. I0703 14:21:08.911675 1 tasks_processing.go:71] worker 14 working on active_alerts task. I0703 14:21:08.912135 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=c6f89ae61f9d123cea81cf60163a968c8852204a818fe7f969c9fed2534156b5 I0703 14:21:08.912321 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=579ae8c1aa65f3d508fc7354470843d1e74015e3869691fa636363900f1d3c6e I0703 14:21:08.912460 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=18fc19b2be7a1352e00cb1ee390b1bb89fce8bab4f02dd659faa833a30fd3952 I0703 14:21:08.912494 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=bba20bef1885304e6c42cb735a190c415ca627e1106775edba44096974be8a68 I0703 14:21:08.912516 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=7190b210539ccf4d6772d922eda5725dee770755367c0ce09f4303400eabab89 I0703 14:21:08.912557 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=90f422611fbbc20e8bb701d2ab1b3b3836db33a17dabc07ca65ec7eadcaea39a I0703 14:21:08.912622 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=1f6c85a0df7c4b19fa3dc4e9a63aa804b4932e6ea53d147549672664314b8aaf I0703 14:21:08.912714 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=3521e737ecec5b3faeaa726aa0f0bb2fc29502da59eb968bc5af61c9f426b3de I0703 14:21:08.912766 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=0696a91a4516e6604ced06035587001854533de7a8962d401a94e77ec8b0c838 I0703 14:21:08.912839 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=23ceff9fa43214e794668e8235dfa5e9770327ffc66b24582e942cd792fdbef8 I0703 14:21:08.912893 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=c775e1597a6ab3de1c2dfd22891f044584723b511cc2ceab6631b9189cc7c310 I0703 14:21:08.912949 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=4f76aba4d808dd6fb5ac629c64ac5d0946aac5bd772026197e78025501b5cf34 I0703 14:21:08.913010 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=bf10cf6432b2a72fc8ca827b264e18054e645edbd4a99dca18c507544db45187 I0703 14:21:08.913023 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 24.029011ms to process 13 records I0703 14:21:08.913037 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 6.744482ms to process 0 records I0703 14:21:08.913047 1 tasks_processing.go:71] worker 16 working on install_plans task. E0703 14:21:08.916480 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:41673->172.30.0.10:53: read: connection refused I0703 14:21:08.916501 1 tasks_processing.go:71] worker 15 working on nodes task. W0703 14:21:08.916507 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. E0703 14:21:08.916511 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:41673->172.30.0.10:53: read: connection refused I0703 14:21:08.916548 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.3454ms to process 0 records E0703 14:21:08.917300 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:48756->172.30.0.10:53: read: connection refused I0703 14:21:08.917317 1 tasks_processing.go:71] worker 14 working on service_accounts task. E0703 14:21:08.917333 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:48756->172.30.0.10:53: read: connection refused I0703 14:21:08.917362 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 5.588393ms to process 0 records I0703 14:21:08.926459 1 tasks_processing.go:71] worker 22 working on lokistack task. E0703 14:21:08.926491 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0703 14:21:08.926639 1 gather.go:177] gatherer "clusterconfig" function "machines" took 20.709973ms to process 0 records I0703 14:21:08.926654 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 20.539576ms to process 0 records I0703 14:21:08.926660 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 17.136063ms to process 0 records I0703 14:21:08.926664 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 19.817057ms to process 0 records I0703 14:21:08.926669 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 20.754187ms to process 0 records I0703 14:21:08.926673 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 15.262578ms to process 0 records I0703 14:21:08.926679 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 36.712685ms to process 0 records I0703 14:21:08.926685 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 20.467612ms to process 0 records I0703 14:21:08.926693 1 tasks_processing.go:71] worker 7 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0703 14:21:08.926710 1 tasks_processing.go:71] worker 4 working on infrastructures task. I0703 14:21:08.927744 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=5e82238c473445a3e478a711395d7ba3f67a443135839364893c442889a7359c I0703 14:21:08.927839 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=b28d740e5629233c1f6f9c23e75b3b26647363d41ffc80cf8372de6fb56a135f I0703 14:21:08.927896 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 33.692079ms to process 2 records I0703 14:21:08.928527 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=cddad7b85684d76d1ba30e4ea7a52b8f56a92418d33d536c228a2bee152e246d I0703 14:21:08.928666 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=b4d40fb1377c4986c909882ee4c01e830dd6c1a5ea5d8adceb0f289eca801a80 I0703 14:21:08.928769 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=973738a560abd24cb073536c508fd23c60a71a4546004d12eff3c19d0172dfaf I0703 14:21:08.928800 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 38.518728ms to process 3 records I0703 14:21:08.929164 1 recorder.go:75] Recording config/featuregate with fingerprint=c10d76907aa7528a8007d4471a633458473db2ac9b112d8aa644dcc229d84f22 I0703 14:21:08.929213 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 23.136097ms to process 1 records I0703 14:21:08.929263 1 tasks_processing.go:71] worker 12 working on tsdb_status task. I0703 14:21:08.929357 1 tasks_processing.go:74] worker 21 stopped. I0703 14:21:08.929408 1 tasks_processing.go:74] worker 9 stopped. E0703 14:21:08.929452 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0703 14:21:08.929482 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 17.8197ms to process 0 records I0703 14:21:08.929512 1 tasks_processing.go:74] worker 0 stopped. I0703 14:21:08.929246 1 tasks_processing.go:74] worker 26 stopped. I0703 14:21:08.929254 1 tasks_processing.go:71] worker 31 working on openstack_version task. I0703 14:21:08.929274 1 tasks_processing.go:74] worker 20 stopped. I0703 14:21:08.929292 1 tasks_processing.go:74] worker 11 stopped. I0703 14:21:08.929302 1 tasks_processing.go:74] worker 6 stopped. I0703 14:21:08.929636 1 tasks_processing.go:74] worker 10 stopped. I0703 14:21:08.930474 1 recorder.go:75] Recording config/network with fingerprint=3f81f5dd4d5fdba42434db7191630a28d2cec768531de87d127dfd6f75224425 I0703 14:21:08.930652 1 gather.go:177] gatherer "clusterconfig" function "networks" took 20.019388ms to process 1 records I0703 14:21:08.932427 1 tasks_processing.go:74] worker 13 stopped. I0703 14:21:08.933044 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=06eb9d73fd86abcde8b222faccdacb866a3da28b58ae56d79e417d078d6082a4 I0703 14:21:08.933099 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 22.749535ms to process 1 records I0703 14:21:08.933139 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 6.257356ms to process 0 records I0703 14:21:08.933174 1 tasks_processing.go:74] worker 22 stopped. I0703 14:21:08.933889 1 tasks_processing.go:74] worker 18 stopped. I0703 14:21:08.933918 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0703 14:21:08.933931 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 27.990117ms to process 1 records I0703 14:21:08.934011 1 tasks_processing.go:74] worker 8 stopped. I0703 14:21:08.934092 1 recorder.go:75] Recording config/ingress with fingerprint=f44c71d80a100b733646047ca36aedfcf2357714a7abaf54e988405a925a5fa1 I0703 14:21:08.934107 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 22.305489ms to process 1 records I0703 14:21:08.935249 1 tasks_processing.go:74] worker 2 stopped. I0703 14:21:08.935267 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 47.338146ms to process 0 records I0703 14:21:08.935772 1 tasks_processing.go:74] worker 4 stopped. E0703 14:21:08.936381 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:49097->172.30.0.10:53: read: connection refused I0703 14:21:08.936569 1 recorder.go:75] Recording config/infrastructure with fingerprint=2564142614d86f0e43edb2f9b6d1ac6dd1ad03d154aec34a74335777b8705343 I0703 14:21:08.936589 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 8.544531ms to process 1 records E0703 14:21:08.936604 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.129.0.10:49097->172.30.0.10:53: read: connection refused I0703 14:21:08.936620 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 7.077994ms to process 0 records I0703 14:21:08.936632 1 tasks_processing.go:74] worker 12 stopped. I0703 14:21:08.937582 1 tasks_processing.go:74] worker 31 stopped. I0703 14:21:08.937604 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 7.618984ms to process 0 records E0703 14:21:08.940461 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27ea31dd92-da5b-41ce-b746-557a0dc958ac%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:46029->172.30.0.10:53: read: connection refused I0703 14:21:08.940474 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27ea31dd92-da5b-41ce-b746-557a0dc958ac%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:46029->172.30.0.10:53: read: connection refused I0703 14:21:08.940785 1 tasks_processing.go:74] worker 24 stopped. I0703 14:21:08.941505 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=e493b3372e1b2d819db2460bbe1a891475b9f787e82ef9669b16539f4ff34b63 I0703 14:21:08.942057 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=6e439e6b8596d4cbb704b26964ea903c3e7d1c85e17da814b267bc868997511f I0703 14:21:08.942081 1 gather.go:177] gatherer "clusterconfig" function "crds" took 35.49448ms to process 2 records I0703 14:21:08.943567 1 tasks_processing.go:74] worker 19 stopped. I0703 14:21:08.943628 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 33.832923ms to process 0 records I0703 14:21:08.945598 1 tasks_processing.go:74] worker 1 stopped. I0703 14:21:08.945656 1 recorder.go:75] Recording config/olm_operators with fingerprint=5707307d55aac28ba9024a756efc04a533048a4dcd51ccbd97e8d3e3b8480f5b I0703 14:21:08.945668 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 35.721503ms to process 1 records I0703 14:21:08.947146 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0703 14:21:08.948188 1 tasks_processing.go:74] worker 27 stopped. I0703 14:21:08.948383 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=9b57d1c06b77f23c0f57a330a4c3ba3986465ed9090c193153cfa550a7ea2e3d I0703 14:21:08.948480 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=58efbc8f69c5e08533cf2533340e92c564c4956100dc2bec3f1e0b6917f9fa8f I0703 14:21:08.948492 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 42.712742ms to process 2 records W0703 14:21:08.950854 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:41848->172.30.0.10:53: read: connection refused I0703 14:21:08.950869 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.10:41848->172.30.0.10:53: read: connection refused I0703 14:21:08.954285 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0703 14:21:08.954295 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0703 14:21:08.954308 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0703 14:21:08.957885 1 base_controller.go:82] Caches are synced for ConfigController I0703 14:21:08.957898 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0703 14:21:08.958740 1 tasks_processing.go:74] worker 29 stopped. E0703 14:21:08.958757 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0703 14:21:08.958765 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0703 14:21:08.958772 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0703 14:21:08.958784 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0703 14:21:08.958836 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0703 14:21:08.958845 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0703 14:21:08.958851 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0703 14:21:08.958864 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0703 14:21:08.958949 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0703 14:21:08.958961 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0703 14:21:08.958969 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 70.469801ms to process 7 records I0703 14:21:08.962768 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 14:21:08.995132 1 base_controller.go:82] Caches are synced for LoggingSyncer I0703 14:21:08.995149 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0703 14:21:09.001999 1 tasks_processing.go:74] worker 23 stopped. I0703 14:21:09.002838 1 recorder.go:75] Recording config/version with fingerprint=e109acecfe3af5bac41dacbcb757ff3c4a20fd58bf2ffa8d3afb3e3480507e0d I0703 14:21:09.002919 1 recorder.go:75] Recording config/id with fingerprint=f4b39b3bade42279dfdc0ce8ea318227ea918934b49faf9129c0fb904386edef I0703 14:21:09.002952 1 gather.go:177] gatherer "clusterconfig" function "version" took 114.59644ms to process 2 records I0703 14:21:09.012379 1 tasks_processing.go:74] worker 15 stopped. I0703 14:21:09.012662 1 recorder.go:75] Recording config/node/ip-10-0-0-84.ec2.internal with fingerprint=32eeb1a2b0251c920d530e622960ec37f71c4c3e21f2dc9b95dd9c3e5cb011a2 I0703 14:21:09.012722 1 recorder.go:75] Recording config/node/ip-10-0-1-244.ec2.internal with fingerprint=9065fe6fb8310b224cc9a97ad1258a1f651b415fa755cb9679178c9053f646f5 I0703 14:21:09.012803 1 recorder.go:75] Recording config/node/ip-10-0-2-102.ec2.internal with fingerprint=68ca8c9d09f31c446b396ef78f3c0e2c44c2c69532664e11b33f9788c4488785 I0703 14:21:09.012818 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 95.863331ms to process 3 records I0703 14:21:09.017264 1 tasks_processing.go:74] worker 30 stopped. I0703 14:21:09.017306 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0703 14:21:09.017326 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 127.200452ms to process 1 records I0703 14:21:09.036132 1 tasks_processing.go:74] worker 7 stopped. I0703 14:21:09.036151 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 109.423501ms to process 0 records I0703 14:21:09.037385 1 tasks_processing.go:74] worker 5 stopped. I0703 14:21:09.038684 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm with fingerprint=3a29515ee2863c30388c4bed1599ec289b7956388599149f35f5dd65e288e1b1 I0703 14:21:09.038777 1 recorder.go:75] Recording config/running_containers with fingerprint=1b9b3c7495b1430b51d2556e07a73df25edefea80931e514acbf4f2a0fb0f98e I0703 14:21:09.038792 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 148.970912ms to process 2 records I0703 14:21:09.043880 1 tasks_processing.go:74] worker 17 stopped. E0703 14:21:09.043895 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0703 14:21:09.043901 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ran2luaqta9b05llfbm25d0jj3kitj7-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ran2luaqta9b05llfbm25d0jj3kitj7-primary-cert-bundle-secret" not found I0703 14:21:09.043955 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=7468ffb2ec475da5a77eba0f99fdef0d111b10d58cf9a5a202fb13bd72232784 I0703 14:21:09.043967 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 133.250011ms to process 1 records W0703 14:21:09.915764 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 14:21:10.746544 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0703 14:21:10.915685 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 14:21:10.920296 1 gather_cluster_operator_pods_and_events.go:121] Found 45 pods with 93 containers I0703 14:21:10.920309 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 270600 bytes I0703 14:21:10.920360 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:11.142261 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:11.325932 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:11.350579 1 tasks_processing.go:74] worker 3 stopped. I0703 14:21:11.350690 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=ec672b3d8caf4ef65fe2eeeb7c354f659e9fed9384315d299d99adc305c66a08 I0703 14:21:11.350761 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0703 14:21:11.350788 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=6451049a0f1a666d84f57ae10c9b6753b25f5dab3e7a26b1cd04dd8df24890d9 I0703 14:21:11.350799 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0703 14:21:11.350820 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=6ed1d168070680ab51aaf374d6fa11b3920316190ba1d8cb4ed4b60cbca93c48 I0703 14:21:11.350838 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0703 14:21:11.350860 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=06b961bef432b6982237ff13a339344cd7ab2bbfb9ec053d28e003c497bc0faa I0703 14:21:11.350888 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=cda9cbb67b5400d21a7d92781c2f71b884e5e8caca6be71c5e5b31849978a87a I0703 14:21:11.350913 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=907d4f4518a50e0290cc982eaab23444891781027e3d0e70ccf56a15cfbd617c I0703 14:21:11.350944 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=005f0290c99bde881360e5bdaf2237504b043a9c9e7808e3d20b28ed52e1aa1b I0703 14:21:11.351212 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=d56b938e3dd28319826e6f053515d6d1055e375f8fe9608e86b0f098b3af394f I0703 14:21:11.351252 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=cee9f1c1233deb0cf50e91c56ae48e4afa5b1f91d2b7fbef657e54ca26c9e401 I0703 14:21:11.351263 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0703 14:21:11.351281 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=443f7cba17a4c073c15c62165ff2a9e76a5add1cc9df1d1c2d43875e03cb8d3d I0703 14:21:11.351292 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0703 14:21:11.351307 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=c830fe37299707e2b2cb47b0007cb62b552a1f26f11798c77c1c3b033ab15c62 I0703 14:21:11.351314 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0703 14:21:11.351335 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=92e5fd11c8dac633c765720955584e744a1fec842a6226be90373e3bbd3b9349 I0703 14:21:11.351343 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0703 14:21:11.351363 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=e3e0d1089142ddb1735af55697df54ca3a290e885c8806c365a3c993e8fde433 I0703 14:21:11.351462 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=350218e36c8e11d15cc1b680d5966b97c5fd7cbe13d1749e7a1a7a4b80f64ac2 I0703 14:21:11.351472 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0703 14:21:11.351479 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0703 14:21:11.351501 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0703 14:21:11.351522 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=4825c60a500bea0abed14b6aaf8e70eee25f8e773cd8b679d7c49ca4021e9930 I0703 14:21:11.351545 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=3b751e706f46a3c0d1404694788b6ea756206c5d8f3d479e962acf494be25b2f I0703 14:21:11.351554 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0703 14:21:11.351569 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=559649d8e88c69037dc432eac02c64329f82a5f183c9b41b23b7a4eed9f0601a I0703 14:21:11.351578 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0703 14:21:11.351592 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=d48c192644914f538eae2f47fb91bfeafce99b14fdf05162e6eba77f2ea9ac65 I0703 14:21:11.351607 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=91981960c0148bde37f4e74d0f2b8df75cdfa8b37a09ed4d4f9c8fa824e6fe06 I0703 14:21:11.351620 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=82275a6e8ea010ab58037cb87beebc5f43b317d20a63f50c49c9268dcd1b5bcd I0703 14:21:11.351635 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=fd12a1fc2ab4701e3bd0753f1571450ba6dd7bc5ab757577b01d8b5eb7c9c219 I0703 14:21:11.351655 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=2307f5504016e42aad91076180d15ee595dd216d03be9ef905b6bee62e8ff28b I0703 14:21:11.351665 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0703 14:21:11.351693 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=844c7a5371c2a682a82c386874a9b006e25744c6a18ba3a10ddcc214047fcf59 I0703 14:21:11.351709 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0703 14:21:11.351718 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0703 14:21:11.351725 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.463459545s to process 38 records I0703 14:21:11.525490 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:11.725145 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). W0703 14:21:11.914792 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 14:21:11.925041 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:12.125907 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:12.326027 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:12.543588 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:12.725741 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). W0703 14:21:12.914770 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0703 14:21:12.924975 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:13.126623 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:13.327524 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:13.525659 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:13.724940 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-monitoring-operator container cluster-monitoring-operator-698978dbbf-sp48t pod in namespace openshift-monitoring (previous: false). W0703 14:21:13.915485 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0703 14:21:13.915513 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0703 14:21:13.915531 1 tasks_processing.go:74] worker 25 stopped. E0703 14:21:13.915540 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0703 14:21:13.915550 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0703 14:21:13.915559 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.026796136s to process 1 records I0703 14:21:13.948079 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-state-metrics container kube-state-metrics-7c87d874d5-8b4p4 pod in namespace openshift-monitoring (previous: false). I0703 14:21:14.126331 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container kube-state-metrics-7c87d874d5-8b4p4 pod in namespace openshift-monitoring (previous: false). I0703 14:21:14.324906 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container kube-state-metrics-7c87d874d5-8b4p4 pod in namespace openshift-monitoring (previous: false). I0703 14:21:14.530147 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-798454646c-2z8gz pod in namespace openshift-monitoring (previous: false). I0703 14:21:14.726701 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-798454646c-68s5x pod in namespace openshift-monitoring (previous: false). I0703 14:21:14.926506 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-67788f859f-wwx82 pod in namespace openshift-monitoring (previous: false). I0703 14:21:15.124709 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-67788f859f-zg5qf pod in namespace openshift-monitoring (previous: false). I0703 14:21:15.334539 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-mtwsq pod in namespace openshift-monitoring (previous: false). I0703 14:21:15.524621 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 14:21:15.524637 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-mtwsq pod in namespace openshift-monitoring (previous: false). I0703 14:21:15.726729 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-mtwsq pod in namespace openshift-monitoring (previous: false). I0703 14:21:15.924942 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-t8ftx pod in namespace openshift-monitoring (previous: false). I0703 14:21:16.125553 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 14:21:16.125571 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-t8ftx pod in namespace openshift-monitoring (previous: false). I0703 14:21:16.327036 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-t8ftx pod in namespace openshift-monitoring (previous: false). I0703 14:21:16.526389 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-wstqc pod in namespace openshift-monitoring (previous: false). I0703 14:21:16.725057 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 14:21:16.725074 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-wstqc pod in namespace openshift-monitoring (previous: false). I0703 14:21:16.926829 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-wstqc pod in namespace openshift-monitoring (previous: false). I0703 14:21:17.125409 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container openshift-state-metrics-6cb79d4dbd-9m9r6 pod in namespace openshift-monitoring (previous: false). I0703 14:21:17.325901 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container openshift-state-metrics-6cb79d4dbd-9m9r6 pod in namespace openshift-monitoring (previous: false). I0703 14:21:17.524747 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for openshift-state-metrics container openshift-state-metrics-6cb79d4dbd-9m9r6 pod in namespace openshift-monitoring (previous: false). I0703 14:21:17.726000 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:17.926132 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:18.127265 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:18.325094 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:18.525946 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:18.725434 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:18.924292 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0703 14:21:19.124537 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:19.326232 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:19.527695 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:19.726552 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:19.926444 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:20.126081 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:20.326211 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0703 14:21:20.526497 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator container prometheus-operator-5df5b8bf6d-7w5sq pod in namespace openshift-monitoring (previous: false). I0703 14:21:20.728714 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-operator-5df5b8bf6d-7w5sq pod in namespace openshift-monitoring (previous: false). I0703 14:21:20.924929 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-68c48fd977-856pd pod in namespace openshift-monitoring (previous: false). I0703 14:21:21.125494 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-68c48fd977-nsr8n pod in namespace openshift-monitoring (previous: false). I0703 14:21:21.325397 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-798f87cc4d-bbh2p pod in namespace openshift-monitoring (previous: false). I0703 14:21:21.526195 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-798f87cc4d-bbh2p pod in namespace openshift-monitoring (previous: false). I0703 14:21:21.724941 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-798f87cc4d-bbh2p pod in namespace openshift-monitoring (previous: false). I0703 14:21:21.925063 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-798f87cc4d-bbh2p pod in namespace openshift-monitoring (previous: false). I0703 14:21:21.939760 1 tasks_processing.go:74] worker 16 stopped. I0703 14:21:21.939802 1 recorder.go:75] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0703 14:21:21.939814 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.026698206s to process 1 records I0703 14:21:22.126552 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-798f87cc4d-bbh2p pod in namespace openshift-monitoring (previous: false). I0703 14:21:22.325148 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-798f87cc4d-bbh2p pod in namespace openshift-monitoring (previous: false). I0703 14:21:22.525694 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-798f87cc4d-m8d7q pod in namespace openshift-monitoring (previous: false). I0703 14:21:22.723623 1 tasks_processing.go:74] worker 14 stopped. I0703 14:21:22.723898 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=d8061d111763aa3c93810c51f1810f5d0175e0bccc3dc52b0a0649d31668ee77 I0703 14:21:22.723915 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.806289527s to process 1 records I0703 14:21:22.726419 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-798f87cc4d-m8d7q pod in namespace openshift-monitoring (previous: false). I0703 14:21:22.925241 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-798f87cc4d-m8d7q pod in namespace openshift-monitoring (previous: false). I0703 14:21:23.132304 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-798f87cc4d-m8d7q pod in namespace openshift-monitoring (previous: false). I0703 14:21:23.325286 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-798f87cc4d-m8d7q pod in namespace openshift-monitoring (previous: false). I0703 14:21:23.527307 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-798f87cc4d-m8d7q pod in namespace openshift-monitoring (previous: false). I0703 14:21:23.725772 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-nk4td pod in namespace openshift-network-console (previous: false). I0703 14:21:23.925602 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-zlr59 pod in namespace openshift-network-console (previous: false). I0703 14:21:24.126776 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-xv4dm pod in namespace openshift-console-operator (previous: true). I0703 14:21:24.325573 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-xv4dm pod in namespace openshift-console-operator (previous: false). I0703 14:21:24.529126 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-6b5747df85-bfr65 pod in namespace openshift-console (previous: false). I0703 14:21:24.725075 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-6b5747df85-mt5h2 pod in namespace openshift-console (previous: false). I0703 14:21:24.925736 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-78bf49f587-4869v pod in namespace openshift-console (previous: false). I0703 14:21:25.126003 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-2lcds pod in namespace openshift-console (previous: false). I0703 14:21:25.324947 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-pzhfv pod in namespace openshift-console (previous: false). I0703 14:21:25.525237 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-j5vlv pod in namespace openshift-dns (previous: false). I0703 14:21:25.724679 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-j5vlv pod in namespace openshift-dns (previous: false). I0703 14:21:25.973313 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-mblzd pod in namespace openshift-dns (previous: false). I0703 14:21:26.134532 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mblzd pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-mblzd\" is waiting to start: ContainerCreating" I0703 14:21:26.134551 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-mblzd\" is waiting to start: ContainerCreating" I0703 14:21:26.134560 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-mblzd pod in namespace openshift-dns (previous: false). I0703 14:21:26.325760 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-mblzd pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-mblzd\" is waiting to start: ContainerCreating" I0703 14:21:26.325779 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-mblzd\" is waiting to start: ContainerCreating" I0703 14:21:26.325818 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-zfpwl pod in namespace openshift-dns (previous: false). I0703 14:21:26.524546 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-zfpwl pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-zfpwl\" is waiting to start: ContainerCreating" I0703 14:21:26.524564 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-zfpwl\" is waiting to start: ContainerCreating" I0703 14:21:26.524575 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-zfpwl pod in namespace openshift-dns (previous: false). I0703 14:21:26.724746 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-zfpwl pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-zfpwl\" is waiting to start: ContainerCreating" I0703 14:21:26.724764 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-zfpwl\" is waiting to start: ContainerCreating" I0703 14:21:26.724778 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-fh8lj pod in namespace openshift-dns (previous: false). I0703 14:21:26.925326 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 14:21:26.925344 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-l7gwm pod in namespace openshift-dns (previous: false). I0703 14:21:27.125059 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 14:21:27.125077 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-sk8c5 pod in namespace openshift-dns (previous: false). I0703 14:21:27.324262 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0703 14:21:27.324282 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-54c547dd64-h2xcp pod in namespace openshift-image-registry (previous: false). I0703 14:21:27.525477 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6f66f56747-fzmvr pod in namespace openshift-image-registry (previous: false). I0703 14:21:27.730088 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6f66f56747-spzp4 pod in namespace openshift-image-registry (previous: false). I0703 14:21:27.932983 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9z9jz pod in namespace openshift-image-registry (previous: false). I0703 14:21:28.125455 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-nx4b4 pod in namespace openshift-image-registry (previous: false). I0703 14:21:28.325151 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-qvqm4 pod in namespace openshift-image-registry (previous: false). I0703 14:21:28.525889 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-84d6998647-x54zv pod in namespace openshift-ingress (previous: false). I0703 14:21:28.725803 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-86b97cf84-kkcv5 pod in namespace openshift-ingress (previous: false). I0703 14:21:28.928322 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-86b97cf84-ld22k pod in namespace openshift-ingress (previous: false). I0703 14:21:29.127449 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-57mzj pod in namespace openshift-ingress-canary (previous: false). I0703 14:21:29.324743 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-j6qx7 pod in namespace openshift-ingress-canary (previous: false). I0703 14:21:29.528035 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-j6qx7 pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-j6qx7\" is waiting to start: ContainerCreating" I0703 14:21:29.528055 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-j6qx7\" is waiting to start: ContainerCreating" I0703 14:21:29.528086 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-t7q2d pod in namespace openshift-ingress-canary (previous: false). I0703 14:21:29.724449 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-t7q2d pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t7q2d\" is waiting to start: ContainerCreating" I0703 14:21:29.724469 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-t7q2d\" is waiting to start: ContainerCreating" I0703 14:21:29.724494 1 tasks_processing.go:74] worker 28 stopped. I0703 14:21:29.725148 1 recorder.go:75] Recording events/openshift-monitoring with fingerprint=e7de7be295fe18a0b678904793bccc7bb49dc42b092f2aba0a9ded1e00c5f9e5 I0703 14:21:29.725193 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=3285c12c99f0f87daba4e49f5aa1ced12845a2b0d4e43a2d48dd63c8db1d223d I0703 14:21:29.725341 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=702cd77f49baeeabd54424048bd4b93d40c184f54e85c7681c6529d973300fc1 I0703 14:21:29.725437 1 recorder.go:75] Recording events/openshift-console with fingerprint=80b44db75c30c566fb5d837252074b02b655cfd3f20effd1bc6a7d08a2978e97 I0703 14:21:29.725461 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=b7d2af97720a005172ec352275e384fa069655a7d7a6fb01c3f9171558486b43 I0703 14:21:29.725514 1 recorder.go:75] Recording events/openshift-dns with fingerprint=eacbceff204535cae312eb242cd6e292f6fab6215924b2c682863fa209aa4d6d I0703 14:21:29.725647 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=d11a5d7135109d30c4f65bd66463c16177f52f21205e764bf4a238ae3f25794d I0703 14:21:29.725674 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=48ad06a677c772d92d003a451a0b69e11fb955198f863397edbad124a1541db8 I0703 14:21:29.725742 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=8fccb1f034c29f104f800845bbe4752ded2027bf3bd8854fdb26992d70b44459 I0703 14:21:29.725766 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=667ab93c8f29cbf3145ad392c74fa5dbefc6096165ca276ee32b87795217d42f I0703 14:21:29.725803 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/init-config-reloader_current.log with fingerprint=75dd31e6db3e48b0d67c2ef9c396ce46678f49c699324e1e94cc9c8c7c7119fe I0703 14:21:29.725838 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/alertmanager_current.log with fingerprint=b89d994aa7cfad1a409cc88ac9c763fa951eef6e5f0fe623d45db47894e60d35 I0703 14:21:29.725852 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/config-reloader_current.log with fingerprint=9f7b3885b9281afabbd398b1776952d93bd88254ba9f815217caf0ea78ccc483 I0703 14:21:29.725860 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-web_current.log with fingerprint=e3118ce39c36939a261e064ad70ca4d36ef128b53a134113e9cbcc93aeedf087 I0703 14:21:29.725866 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy_current.log with fingerprint=a0dec5519d6db590e80ab556629a6fe0db2046fd1d5e9cec7e441d6b24cc94ec I0703 14:21:29.725872 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-metric_current.log with fingerprint=d0d22a126ff4a649260d1f9da7a9a9fde493ec2c6c4bdb6f9d648e3f150084f2 I0703 14:21:29.725876 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/prom-label-proxy_current.log with fingerprint=2d8a8d923fd02fe1447419e890722a44a996a26fa068173cdbbdde2dfc681a59 I0703 14:21:29.725882 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/init-config-reloader_current.log with fingerprint=ab0ebed8d27857a798e61b44d4ea1b9916db01ca55c1084ad99115001215a96d I0703 14:21:29.725899 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/alertmanager_current.log with fingerprint=6ea7d18f200734d0fdbace13e9e68e271ce131b71de574881e263ed560520c90 I0703 14:21:29.725912 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/config-reloader_current.log with fingerprint=0dac6a319c75149071f11cc625cadd65eb85dfffce8cf44fa16892596a225fa1 I0703 14:21:29.725918 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-web_current.log with fingerprint=528daf2e390568a8e7f42804d406dfcf17a66e6d027454154ac0fa3cc674f9ae I0703 14:21:29.725924 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy_current.log with fingerprint=c20b5261a689a352b723d6641019aeba4d4a5cbcdab3b359a27c004ff69afb30 I0703 14:21:29.725929 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-metric_current.log with fingerprint=9b4e2e7d08ff1c1eaf42fb8cd29aae94163ddd428d90f740831796d396c42ced I0703 14:21:29.725933 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/prom-label-proxy_current.log with fingerprint=5d6e0b74dfe677bf6c659e687a2c8b3bb4ae28cb7654d115696cc87f7f18c869 I0703 14:21:29.726048 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/cluster-monitoring-operator-698978dbbf-sp48t/cluster-monitoring-operator_current.log with fingerprint=2a57570500c61fab65c2a1523caa41b324487b69be848ed83ee02320a87fadd1 I0703 14:21:29.726070 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-8b4p4/kube-state-metrics_current.log with fingerprint=fb9fe9ca91efbae0e4d6f9797c4ea4b3bc8736df05a9cc5abee5c84c7497262b I0703 14:21:29.726079 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-8b4p4/kube-rbac-proxy-main_current.log with fingerprint=2aa543f58f51986df78f1c492a0ea4ca096ef0675ffef5efdd30fee8043a1a43 I0703 14:21:29.726085 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-8b4p4/kube-rbac-proxy-self_current.log with fingerprint=16066086f65602d8982814afccaac988a1ce06f474c9aa4cc5e5c5f4536a5b6b I0703 14:21:29.726092 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-798454646c-2z8gz/metrics-server_current.log with fingerprint=6b011e7f0da256d5a85787a4c0f862e8375f3451be2785d75adc3f79c0571ece I0703 14:21:29.726100 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-798454646c-68s5x/metrics-server_current.log with fingerprint=694665370fba05e352b9f7f637d1861b0b05d5cb54db42ee0eae963d374bb023 I0703 14:21:29.726110 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-67788f859f-wwx82/monitoring-plugin_current.log with fingerprint=57e2374ad69a117cfe58bba6e112d0ba68ff2cbd12832f0ad625d424d82ced0e I0703 14:21:29.726123 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-67788f859f-zg5qf/monitoring-plugin_current.log with fingerprint=c6ce92df11b4da9aea6edcd077e07fdde46025a004a5a23427c36b2bacc3945c I0703 14:21:29.726152 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-mtwsq/node-exporter_current.log with fingerprint=d084679dbc484d7d1673fcca5309d39e22721c1ec57494d1e70d811a3c7f05f6 I0703 14:21:29.726160 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-mtwsq/kube-rbac-proxy_current.log with fingerprint=2857ee259c6b7ee8a82d9e996b7217cefb771c1f3c1be17c72478c4c068d75ee I0703 14:21:29.726186 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-t8ftx/node-exporter_current.log with fingerprint=8433fd0d2c6ee892a3eb19d4160507bdcc9b5144212af7b08c8cfc5d7beaee48 I0703 14:21:29.726194 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-t8ftx/kube-rbac-proxy_current.log with fingerprint=2fb920fde0886a3eb8830be3c7f7c4a8d9859263e38ed4c9a8147e90f3257ca9 I0703 14:21:29.726231 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-wstqc/node-exporter_current.log with fingerprint=8ff1741c71134412caf99590d275d30a89dca006f46761b0a0910cdd4a647e08 I0703 14:21:29.726241 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-wstqc/kube-rbac-proxy_current.log with fingerprint=83d78512763203ae65f1370d5a6d19b111ec798d0f3a6ae19facf1d0282051a3 I0703 14:21:29.726247 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-9m9r6/kube-rbac-proxy-main_current.log with fingerprint=866249d4ef20670b1c9ed784fb135e33bb8d032ec56ae1ed035afb159c52adfb I0703 14:21:29.726253 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-9m9r6/kube-rbac-proxy-self_current.log with fingerprint=74004f9c11129723ed00b7f0568203d8f570d0259e080d4d9dbe7d467a6f49fd I0703 14:21:29.726263 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-9m9r6/openshift-state-metrics_current.log with fingerprint=155e19d3597dde2d3c63b765d9ebd03ffeba796af1db6e5f7450ddfe464ac0ec I0703 14:21:29.726271 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/init-config-reloader_current.log with fingerprint=f47bbd116cfa39e3a3a43b77b5a36f27e841d16a0ed22a410e75b88507a0be56 I0703 14:21:29.726365 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/prometheus_current.log with fingerprint=e2b045cbbce5d9c5b17c413c62216ef5fff0960013f03187f1b01d92c5090695 I0703 14:21:29.726379 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/config-reloader_current.log with fingerprint=036b02c169114fca182ccd7dd967e0c4431c7810483494e6566e5b4cda1421a8 I0703 14:21:29.726390 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/thanos-sidecar_current.log with fingerprint=12a62dd1e2a9666edd4eb8e775a1bfa12f419591e5e912785af33d0b90a0d2aa I0703 14:21:29.726399 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-web_current.log with fingerprint=0d118739a6f501041e032176406ca37d7617b5095ecdb37c385289733198a02b I0703 14:21:29.726407 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy_current.log with fingerprint=07b07e4411d9fa527f50ab4ceadb57cba52aa6677af3e8e8f4745c98c54eade5 I0703 14:21:29.726413 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-thanos_current.log with fingerprint=4ec0f9f4b2c27440e22feb516c97dd5df85d0af5f3f3e3cc3432203417d68c76 I0703 14:21:29.726419 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/init-config-reloader_current.log with fingerprint=42b09793da7692f0621eaf2675c4e4cf8c8888eb63e222d3f83021d4263086ff I0703 14:21:29.726512 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/prometheus_current.log with fingerprint=242b4a3483c7e6eea4b97263cd4c97dca1e77330ebc418f58be48cb48c679b2c I0703 14:21:29.726527 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/config-reloader_current.log with fingerprint=56f524bcc822a19d77354e9d18324fc72e4a1b70e053c186f6b6245697b4ca91 I0703 14:21:29.726539 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/thanos-sidecar_current.log with fingerprint=68784d03ae3aaf10b027dcc019581c431a30b39057f5afb5369f0f1d215d8814 I0703 14:21:29.726545 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-web_current.log with fingerprint=b0fc804c565c818e3bad30d752160e42e63344f1ec80da539b9772d6b2cba556 I0703 14:21:29.726551 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy_current.log with fingerprint=835e2582450753f906d0a7f3d19bfa50b0f5d05876774ffab8ac52244db17a22 I0703 14:21:29.726557 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-thanos_current.log with fingerprint=db56cf5158b176d7cfaf0be2581d063b3d76fb121260ded5b7f72e17ea40e686 I0703 14:21:29.726635 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-5df5b8bf6d-7w5sq/prometheus-operator_current.log with fingerprint=56550ecacc97fa0efa8f774c1a24618a032d22412e7fc36c8d302281b5a2a1ae I0703 14:21:29.726644 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-5df5b8bf6d-7w5sq/kube-rbac-proxy_current.log with fingerprint=6df4546d8d638f277ecb1054d85826c1bfd9cf1912a7ba4bc0006646cd79dd21 I0703 14:21:29.726651 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-68c48fd977-856pd/prometheus-operator-admission-webhook_current.log with fingerprint=5a2227bcf572444468472f1882cc3025fdb7c172b204d7f6da28bc62b0b7fee7 I0703 14:21:29.726658 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-68c48fd977-nsr8n/prometheus-operator-admission-webhook_current.log with fingerprint=3db174586c70de2add16462ab211813c672947f3ce79198a63b7042eb5be7dcd I0703 14:21:29.726683 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-bbh2p/thanos-query_current.log with fingerprint=f215860cb9855bc8cd55201a904e645ee1b18e460c54b1796343856050969c42 I0703 14:21:29.726694 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-bbh2p/kube-rbac-proxy-web_current.log with fingerprint=4152f654992b8045f8cabdac713913c5aca2d63335157af88f672876961c2a40 I0703 14:21:29.726700 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-bbh2p/kube-rbac-proxy_current.log with fingerprint=a88d52541cd8485e62e2d66fff7cee22f86b573fc28e0f86a19b0dd5bf356123 I0703 14:21:29.726704 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-bbh2p/prom-label-proxy_current.log with fingerprint=3b4cf92dabe1bc86400817703ad77d86d53231bf7a9fd8c16e8d2ac366f1dbe8 I0703 14:21:29.726709 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-bbh2p/kube-rbac-proxy-rules_current.log with fingerprint=424ded955d3044859563f76321e46427c4e1b4ed3c3bac00bd91ef8071800d91 I0703 14:21:29.726717 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-bbh2p/kube-rbac-proxy-metrics_current.log with fingerprint=8787d912c0141a9aba2bccf425f9d098393563a05ddb2d1b7f21e5fa4c5c0396 I0703 14:21:29.726739 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-m8d7q/thanos-query_current.log with fingerprint=d96232ee33586a057b5655c02ea5a989413364f488693da4bb7e2db7a3571dd9 I0703 14:21:29.726753 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-m8d7q/kube-rbac-proxy-web_current.log with fingerprint=789994d65267f7e1af1ab1de4f5a1f4cebd660cfa4baff9b58047dca2177e168 I0703 14:21:29.726758 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-m8d7q/kube-rbac-proxy_current.log with fingerprint=420dcdc5bcee12147e2ebebe5e0ad477176b4e1a5c0721236d12e3985790ea16 I0703 14:21:29.726763 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-m8d7q/prom-label-proxy_current.log with fingerprint=055213d5189d816e5c7daa199854a31477f465b04fcd4979fb6e853dd6a429af I0703 14:21:29.726768 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-m8d7q/kube-rbac-proxy-rules_current.log with fingerprint=2cdca53e1763785c9737886ef3320ed22a62e078b42de041c0b2dd8498146913 I0703 14:21:29.726774 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-798f87cc4d-m8d7q/kube-rbac-proxy-metrics_current.log with fingerprint=c5fe58bbbc37eb3fdae1548fecf403f817906e95aedcb53042fa3e2fe7591d14 I0703 14:21:29.726780 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-nk4td/networking-console-plugin_current.log with fingerprint=7213a211f546d8fb504e37bfff3b0bf2ab2178da0f85ecdfb76a0d83d9116041 I0703 14:21:29.726786 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-zlr59/networking-console-plugin_current.log with fingerprint=c1a09663c4f2c3f15812058fa43c8810de1a251a56c5333ca58762bc885126c4 I0703 14:21:29.726924 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm with fingerprint=3a29515ee2863c30388c4bed1599ec289b7956388599149f35f5dd65e288e1b1 E0703 14:21:29.726943 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json" was already recorded and had the fingerprint "3a29515ee2863c30388c4bed1599ec289b7956388599149f35f5dd65e288e1b1", overwriting with the record having fingerprint "3a29515ee2863c30388c4bed1599ec289b7956388599149f35f5dd65e288e1b1" W0703 14:21:29.726955 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json" because of the warning: warning: the record with the same fingerprint "3a29515ee2863c30388c4bed1599ec289b7956388599149f35f5dd65e288e1b1" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json" I0703 14:21:29.726973 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-xv4dm/console-operator_previous.log with fingerprint=c4ab211dd34ff930bf8dc65c455d5ca63e7025d3f550e78ed64d35c9a2861b3f I0703 14:21:29.727197 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-xv4dm/console-operator_current.log with fingerprint=b11a03351a0e9c10fe819cfbfc6b5e364929e4c855d4a90832f2b3e5c8cda08a I0703 14:21:29.727212 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-6b5747df85-bfr65/console_current.log with fingerprint=4095b4612a016e76b4885f7521e04e1cb5ed8421841bb269f277be47a4882fc7 I0703 14:21:29.727235 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-6b5747df85-mt5h2/console_current.log with fingerprint=3f35642fda233b6c2645be57c8f25b105a2252cafec2360032cf5882a072ff49 I0703 14:21:29.727249 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-78bf49f587-4869v/console_current.log with fingerprint=1afe3de810cf3512fb8286f1c28b25a2bdf995577780ad73461a5fcf8a6e6424 I0703 14:21:29.727254 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-2lcds/download-server_current.log with fingerprint=035238002c468c82c2835b835e1167521c2e419fcfbe62362b77bb9c45262316 I0703 14:21:29.727260 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-pzhfv/download-server_current.log with fingerprint=0975f0ebd321df659dd020125154171bf913181dd3c46293b6ea43d330b2f8d6 I0703 14:21:29.727387 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-j5vlv with fingerprint=4088f10d00beea921afe492153b7128a36126eb6147acad1784768c239ca278b I0703 14:21:29.727397 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-j5vlv/dns_current.log with fingerprint=306bcc7763047631ac1258c52525607021e965fe028362c4ae07edbbdd4d974a I0703 14:21:29.727403 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-j5vlv/kube-rbac-proxy_current.log with fingerprint=3bd04f8f0be7d6dfe8fa1612e11482c975458eea87b05f076e78a7d5735e7621 I0703 14:21:29.727474 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-mblzd with fingerprint=e64492fb02b2695caaad1570df77c7651f120966306d6741ad581d44b7a8a389 I0703 14:21:29.727546 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-zfpwl with fingerprint=0f0d8246c0f2bda794891905d315bc3d3f004d9d53003ab62f267c47dae228fc I0703 14:21:29.727567 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-54c547dd64-h2xcp/registry_current.log with fingerprint=9cf9734641b2998b299bd94db7ca23b33fa1591317fba920c148a216d7d1ca69 I0703 14:21:29.727609 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-6f66f56747-fzmvr/registry_current.log with fingerprint=c00c79e4cc850fe4c32601c97131305e7ab48aca34bb611b6f90e53cb68d2d6d I0703 14:21:29.727651 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-6f66f56747-spzp4/registry_current.log with fingerprint=ec1da524c4b5d56da371e98b76c2f72d280b08ff41b99a6d2b5845e6eb24fcbe I0703 14:21:29.727658 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-9z9jz/node-ca_current.log with fingerprint=55d6a47bc919fdf167e073ae419eb8f75aa3db02d0a7313e8c278f44717c627b I0703 14:21:29.727664 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-nx4b4/node-ca_current.log with fingerprint=ba3f08f7742dc69ae16f8b202d617821a11676ea95d529a418fe7e24392fecb0 I0703 14:21:29.727668 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-qvqm4/node-ca_current.log with fingerprint=0fc7aeb3a478e3c149e67d595c1e23542111e6369d2c03da9e732e0ac0722688 I0703 14:21:29.727695 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-84d6998647-x54zv/router_current.log with fingerprint=906b7c76b60d825169c0bb3afb48521f9f5c71d982994a1d1f4dfd0096644c63 I0703 14:21:29.727734 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-86b97cf84-kkcv5/router_current.log with fingerprint=ad8f12c83b09f812e85f346eb482bd348c125dc512abd1c77d3175caaab985cd I0703 14:21:29.727773 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-86b97cf84-ld22k/router_current.log with fingerprint=57e909db3a1e1a2c3cdc03e38fed519eb94037f80d67b71f10e6418f7ee8fb13 I0703 14:21:29.727779 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-57mzj/serve-healthcheck-canary_current.log with fingerprint=31d7e656653c20508e1d74b1878ad3201ad458673771e709f2957e1dc6255126 I0703 14:21:29.727837 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-j6qx7 with fingerprint=b74916af74854f89e592fb5dfdb77660a700af8a75f5b462d0946109ae723305 I0703 14:21:29.727891 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-t7q2d with fingerprint=6d092e0189dc49b8c361664a4b0fed3f433838c06120b24e86b0fd9c0fa3ce3c I0703 14:21:29.727900 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 20.836467785s to process 98 records E0703 14:21:29.727969 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 20.84s with: function \"silenced_alerts\" failed with an error, function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"metrics\" failed with an error, function \"active_alerts\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"tsdb_status\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json\"" I0703 14:21:29.729077 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "silenced_alerts" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "tsdb_status" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-xv4dm.json" I0703 14:21:29.729092 1 periodic.go:209] Running workloads gatherer I0703 14:21:29.729107 1 tasks_processing.go:45] number of workers: 2 I0703 14:21:29.729118 1 tasks_processing.go:69] worker 1 listening for tasks. I0703 14:21:29.729124 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0703 14:21:29.729125 1 tasks_processing.go:69] worker 0 listening for tasks. I0703 14:21:29.729195 1 tasks_processing.go:71] worker 0 working on workload_info task. I0703 14:21:29.735561 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-244.ec2.internal... I0703 14:21:29.735568 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-84.ec2.internal... I0703 14:21:29.735577 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-102.ec2.internal... I0703 14:21:29.757919 1 tasks_processing.go:74] worker 1 stopped. I0703 14:21:29.757936 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 28.778308ms to process 0 records I0703 14:21:35.168080 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0703 14:21:35.191847 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0703 14:21:35.200722 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0703 14:21:35.209791 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0703 14:21:35.218951 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (9ms) I0703 14:21:35.228103 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (9ms) I0703 14:21:35.236965 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (9ms) I0703 14:21:35.245370 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (8ms) I0703 14:21:35.254106 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (9ms) I0703 14:21:35.262892 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (9ms) I0703 14:21:35.271429 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (9ms) I0703 14:21:35.280260 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (9ms) I0703 14:21:35.300310 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (20ms) I0703 14:21:35.401254 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (101ms) I0703 14:21:35.501834 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (101ms) I0703 14:21:35.601335 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (99ms) I0703 14:21:35.702251 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0703 14:21:35.802398 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0703 14:21:35.901826 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0703 14:21:36.001569 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0703 14:21:36.101274 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (100ms) I0703 14:21:36.202108 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (101ms) I0703 14:21:36.302291 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (100ms) I0703 14:21:36.401365 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0703 14:21:36.501758 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (100ms) I0703 14:21:36.601668 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0703 14:21:36.702076 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (100ms) I0703 14:21:36.801780 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0703 14:21:36.901873 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0703 14:21:37.001087 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (99ms) I0703 14:21:37.101206 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0703 14:21:37.201261 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (100ms) I0703 14:21:37.301434 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (100ms) I0703 14:21:37.401553 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0703 14:21:37.501312 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0703 14:21:37.600460 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0703 14:21:37.701041 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (101ms) I0703 14:21:37.801175 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0703 14:21:37.900317 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0703 14:21:38.001443 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (101ms) I0703 14:21:38.101830 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (100ms) I0703 14:21:38.201408 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0703 14:21:38.301831 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0703 14:21:38.402170 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0703 14:21:38.501745 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (100ms) I0703 14:21:38.600968 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (99ms) I0703 14:21:38.701524 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0703 14:21:38.801087 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0703 14:21:38.900852 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (100ms) I0703 14:21:39.002464 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (102ms) I0703 14:21:39.103667 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (101ms) I0703 14:21:39.201284 1 gather_workloads_info.go:387] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (98ms) I0703 14:21:39.201330 1 tasks_processing.go:74] worker 0 stopped. I0703 14:21:39.201696 1 recorder.go:75] Recording config/workload_info with fingerprint=b17b2b7317ff789f3ce54978087fdb85b7387a5795064b3273d6c9271a6d9aba I0703 14:21:39.201717 1 gather.go:177] gatherer "workloads" function "workload_info" took 9.472122123s to process 1 records I0703 14:21:39.201745 1 periodic.go:256] Periodic gather workloads completed in 9.472s I0703 14:21:39.201759 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0703 14:21:39.201765 1 periodic.go:209] Running conditional gatherer I0703 14:21:39.209540 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0703 14:21:39.389246 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0703 14:21:39.394907 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.181.70:9091: connect: connection refused I0703 14:21:39.394990 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0703 14:21:39.402537 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0703 14:21:39.402555 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402563 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402569 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402572 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402575 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402577 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402580 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402583 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402586 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0703 14:21:39.402589 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0703 14:21:39.402607 1 tasks_processing.go:45] number of workers: 3 I0703 14:21:39.402616 1 tasks_processing.go:69] worker 2 listening for tasks. I0703 14:21:39.402621 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0703 14:21:39.402639 1 tasks_processing.go:69] worker 1 listening for tasks. I0703 14:21:39.402640 1 tasks_processing.go:69] worker 0 listening for tasks. I0703 14:21:39.402654 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0703 14:21:39.402650 1 tasks_processing.go:71] worker 1 working on remote_configuration task. I0703 14:21:39.402656 1 tasks_processing.go:74] worker 0 stopped. I0703 14:21:39.402731 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0703 14:21:39.402745 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.274µs to process 1 records I0703 14:21:39.402795 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0703 14:21:39.402808 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.555µs to process 1 records I0703 14:21:39.402817 1 tasks_processing.go:74] worker 1 stopped. I0703 14:21:39.402953 1 tasks_processing.go:74] worker 2 stopped. I0703 14:21:39.402963 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 291.13µs to process 0 records I0703 14:21:39.402977 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0703 14:21:39.402991 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0703 14:21:39.430620 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=4382c3f068b8f2db33cb80632fe97f233a9da0c3d432ff60d32ac4c56536d3b1 I0703 14:21:39.430836 1 diskrecorder.go:70] Writing 204 records to /var/lib/insights-operator/insights-2026-07-03-142139.tar.gz I0703 14:21:39.451373 1 diskrecorder.go:51] Wrote 204 records to disk in 20ms I0703 14:21:39.451423 1 periodic.go:278] Gathering cluster info every 2h0m0s I0703 14:21:39.451445 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0703 14:22:38.888104 1 diskrecorder.go:223] Found files to send: insights-2026-07-03-142139.tar.gz I0703 14:22:38.888133 1 insightsuploader.go:150] Checking archives to upload periodically every 15m28.815810025s I0703 14:22:38.888142 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0703 14:22:38.902244 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0703 14:22:39.118571 1 requests.go:87] Successfully reported id=2026-07-03T14:22:38Z x-rh-insights-request-id=63de282d11c34b7f9a5960469b8888cb, wrote=107500 I0703 14:22:39.118640 1 insightsuploader.go:187] Uploaded report successfully in 230.485623ms I0703 14:22:39.118670 1 controller.go:128] Initializing last reported time to 2026-07-03T14:22:38Z I0703 14:22:39.118737 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0703 14:22:39.118748 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0703 14:22:39.118758 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0703 14:22:39.125702 1 controller.go:512] The operator is healthy I0703 14:23:08.902236 1 controller.go:512] The operator is healthy I0703 14:23:44.158688 1 insightsreport.go:137] Pulling report from smart-proxy I0703 14:23:44.158712 1 insightsreport.go:149] Retrieving report I0703 14:23:44.166859 1 requests.go:110] Retrieving report for cluster: ea31dd92-da5b-41ce-b746-557a0dc958ac I0703 14:23:44.166873 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/ea31dd92-da5b-41ce-b746-557a0dc958ac/reports I0703 14:23:44.169626 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/ea31dd92-da5b-41ce-b746-557a0dc958ac/reports I0703 14:23:44.311613 1 insightsreport.go:184] Report retrieved I0703 14:23:44.351974 1 insightsreport.go:239] Report retrieved correctly I0703 14:25:08.904664 1 controller.go:512] The operator is healthy I0703 14:26:08.879021 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0703 14:26:08.888006 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 14:26:08.888027 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0703 14:26:08.892038 1 secretconfigobserver.go:119] support secret does not exist I0703 14:27:08.909727 1 controller.go:512] The operator is healthy I0703 14:29:08.901309 1 controller.go:512] The operator is healthy I0703 14:31:08.876998 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 14:31:08.893075 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0703 14:31:08.898309 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 14:31:08.898331 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0703 14:31:08.901093 1 controller.go:512] The operator is healthy I0703 14:31:08.903022 1 secretconfigobserver.go:119] support secret does not exist I0703 14:33:08.902019 1 controller.go:512] The operator is healthy I0703 14:35:08.902176 1 controller.go:512] The operator is healthy I0703 14:36:08.903938 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0703 14:36:08.909004 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 14:36:08.909037 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0703 14:36:08.913349 1 secretconfigobserver.go:119] support secret does not exist I0703 14:37:08.900928 1 controller.go:512] The operator is healthy I0703 14:38:12.936325 1 insightsuploader.go:146] Nothing to report since 2026-07-03T14:22:38Z I0703 14:39:08.901639 1 controller.go:512] The operator is healthy I0703 14:41:08.876186 1 configmapobserver.go:84] configmaps "insights-config" not found I0703 14:41:08.901184 1 controller.go:512] The operator is healthy I0703 14:41:08.913971 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0703 14:41:08.918873 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0703 14:41:08.918893 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0703 14:41:08.923489 1 secretconfigobserver.go:119] support secret does not exist