I0329 02:55:06.538948 1 cmd.go:253] Using service-serving-cert provided certificates I0329 02:55:06.539447 1 observer_polling.go:159] Starting file observer I0329 02:55:07.082106 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0329 02:55:07.082574 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0329 02:55:07.083804 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0329 02:55:07.084021 1 secure_serving.go:57] Forcing use of http/1.1 only W0329 02:55:07.084038 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0329 02:55:07.084042 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0329 02:55:07.084046 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0329 02:55:07.084050 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0329 02:55:07.084054 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0329 02:55:07.084057 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0329 02:55:07.087357 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0329 02:55:07.087385 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"1cd44cf4-ee9f-4869-bebd-79903f9e93c4", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0329 02:55:07.088163 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0329 02:55:07.088179 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0329 02:55:07.088220 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0329 02:55:07.088261 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0329 02:55:07.088210 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0329 02:55:07.088274 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0329 02:55:07.088448 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0329 02:55:07.088705 1 secure_serving.go:213] Serving securely on [::]:8443 I0329 02:55:07.088731 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0329 02:55:07.092073 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0329 02:55:07.092096 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0329 02:55:07.092193 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0329 02:55:07.097193 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0329 02:55:07.097209 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0329 02:55:07.101395 1 secretconfigobserver.go:119] support secret does not exist I0329 02:55:07.104938 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0329 02:55:07.108332 1 secretconfigobserver.go:119] support secret does not exist I0329 02:55:07.111070 1 recorder.go:161] Pruning old reports every 7h40m34s, max age is 288h0m0s I0329 02:55:07.115293 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0329 02:55:07.115302 1 periodic.go:209] Running clusterconfig gatherer I0329 02:55:07.115316 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0329 02:55:07.115297 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0329 02:55:07.115333 1 insightsreport.go:296] Starting report retriever I0329 02:55:07.115338 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0329 02:55:07.115360 1 tasks_processing.go:45] number of workers: 64 I0329 02:55:07.115381 1 tasks_processing.go:69] worker 2 listening for tasks. I0329 02:55:07.115390 1 tasks_processing.go:69] worker 8 listening for tasks. I0329 02:55:07.115396 1 tasks_processing.go:69] worker 1 listening for tasks. I0329 02:55:07.115397 1 tasks_processing.go:69] worker 3 listening for tasks. I0329 02:55:07.115402 1 tasks_processing.go:69] worker 4 listening for tasks. I0329 02:55:07.115400 1 tasks_processing.go:69] worker 0 listening for tasks. I0329 02:55:07.115408 1 tasks_processing.go:69] worker 14 listening for tasks. I0329 02:55:07.115410 1 tasks_processing.go:69] worker 5 listening for tasks. I0329 02:55:07.115415 1 tasks_processing.go:69] worker 6 listening for tasks. I0329 02:55:07.115413 1 tasks_processing.go:69] worker 17 listening for tasks. I0329 02:55:07.115415 1 tasks_processing.go:69] worker 21 listening for tasks. I0329 02:55:07.115421 1 tasks_processing.go:69] worker 16 listening for tasks. I0329 02:55:07.115423 1 tasks_processing.go:69] worker 7 listening for tasks. I0329 02:55:07.115422 1 tasks_processing.go:69] worker 9 listening for tasks. I0329 02:55:07.115428 1 tasks_processing.go:71] worker 16 working on metrics task. I0329 02:55:07.115429 1 tasks_processing.go:71] worker 21 working on schedulers task. I0329 02:55:07.115433 1 tasks_processing.go:69] worker 34 listening for tasks. I0329 02:55:07.115429 1 tasks_processing.go:69] worker 18 listening for tasks. I0329 02:55:07.115432 1 tasks_processing.go:69] worker 11 listening for tasks. I0329 02:55:07.115433 1 tasks_processing.go:69] worker 15 listening for tasks. I0329 02:55:07.115443 1 tasks_processing.go:69] worker 22 listening for tasks. I0329 02:55:07.115437 1 tasks_processing.go:69] worker 12 listening for tasks. I0329 02:55:07.115444 1 tasks_processing.go:69] worker 20 listening for tasks. I0329 02:55:07.115450 1 tasks_processing.go:69] worker 28 listening for tasks. I0329 02:55:07.115447 1 tasks_processing.go:69] worker 47 listening for tasks. I0329 02:55:07.115457 1 tasks_processing.go:69] worker 24 listening for tasks. I0329 02:55:07.115457 1 tasks_processing.go:69] worker 40 listening for tasks. I0329 02:55:07.115459 1 tasks_processing.go:69] worker 30 listening for tasks. I0329 02:55:07.115462 1 tasks_processing.go:69] worker 27 listening for tasks. I0329 02:55:07.115464 1 tasks_processing.go:69] worker 26 listening for tasks. I0329 02:55:07.115459 1 tasks_processing.go:71] worker 47 working on authentication task. I0329 02:55:07.115467 1 tasks_processing.go:71] worker 27 working on nodes task. I0329 02:55:07.115464 1 tasks_processing.go:71] worker 28 working on openstack_dataplanedeployments task. I0329 02:55:07.115473 1 tasks_processing.go:69] worker 32 listening for tasks. I0329 02:55:07.115477 1 tasks_processing.go:71] worker 32 working on sap_pods task. I0329 02:55:07.115478 1 tasks_processing.go:69] worker 25 listening for tasks. I0329 02:55:07.115479 1 tasks_processing.go:69] worker 60 listening for tasks. I0329 02:55:07.115488 1 tasks_processing.go:69] worker 45 listening for tasks. I0329 02:55:07.115485 1 tasks_processing.go:69] worker 37 listening for tasks. I0329 02:55:07.115492 1 tasks_processing.go:69] worker 38 listening for tasks. I0329 02:55:07.115449 1 tasks_processing.go:69] worker 23 listening for tasks. I0329 02:55:07.115454 1 tasks_processing.go:69] worker 29 listening for tasks. I0329 02:55:07.115501 1 tasks_processing.go:69] worker 48 listening for tasks. I0329 02:55:07.115467 1 tasks_processing.go:69] worker 31 listening for tasks. I0329 02:55:07.115504 1 tasks_processing.go:69] worker 61 listening for tasks. I0329 02:55:07.115509 1 tasks_processing.go:69] worker 39 listening for tasks. I0329 02:55:07.115509 1 tasks_processing.go:71] worker 18 working on pod_network_connectivity_checks task. I0329 02:55:07.115435 1 tasks_processing.go:69] worker 19 listening for tasks. I0329 02:55:07.115518 1 tasks_processing.go:69] worker 62 listening for tasks. I0329 02:55:07.115444 1 tasks_processing.go:69] worker 13 listening for tasks. I0329 02:55:07.115525 1 tasks_processing.go:71] worker 0 working on ingress_certificates task. I0329 02:55:07.115524 1 tasks_processing.go:69] worker 46 listening for tasks. I0329 02:55:07.115530 1 tasks_processing.go:71] worker 15 working on machine_healthchecks task. I0329 02:55:07.115525 1 tasks_processing.go:71] worker 22 working on monitoring_persistent_volumes task. I0329 02:55:07.115534 1 tasks_processing.go:71] worker 14 working on version task. I0329 02:55:07.115536 1 tasks_processing.go:71] worker 17 working on node_logs task. I0329 02:55:07.115540 1 tasks_processing.go:71] worker 8 working on validating_webhook_configurations task. I0329 02:55:07.115540 1 tasks_processing.go:71] worker 1 working on openstack_version task. I0329 02:55:07.115543 1 tasks_processing.go:69] worker 53 listening for tasks. I0329 02:55:07.115546 1 tasks_processing.go:71] worker 3 working on machine_sets task. I0329 02:55:07.115549 1 tasks_processing.go:71] worker 60 working on cost_management_metrics_configs task. I0329 02:55:07.115552 1 tasks_processing.go:71] worker 53 working on jaegers task. I0329 02:55:07.115570 1 tasks_processing.go:69] worker 56 listening for tasks. I0329 02:55:07.115592 1 tasks_processing.go:69] worker 54 listening for tasks. I0329 02:55:07.115587 1 tasks_processing.go:71] worker 45 working on tsdb_status task. I0329 02:55:07.115470 1 tasks_processing.go:71] worker 26 working on container_runtime_configs task. I0329 02:55:07.115601 1 tasks_processing.go:69] worker 55 listening for tasks. I0329 02:55:07.115607 1 tasks_processing.go:69] worker 58 listening for tasks. I0329 02:55:07.115609 1 tasks_processing.go:69] worker 57 listening for tasks. I0329 02:55:07.115545 1 tasks_processing.go:71] worker 25 working on machine_configs task. I0329 02:55:07.115620 1 tasks_processing.go:69] worker 59 listening for tasks. I0329 02:55:07.115617 1 tasks_processing.go:71] worker 38 working on ingress task. I0329 02:55:07.115628 1 tasks_processing.go:71] worker 29 working on operators task. I0329 02:55:07.115694 1 tasks_processing.go:71] worker 48 working on infrastructures task. I0329 02:55:07.115500 1 tasks_processing.go:71] worker 7 working on ceph_cluster task. I0329 02:55:07.115750 1 tasks_processing.go:71] worker 59 working on certificate_signing_requests task. I0329 02:55:07.115497 1 tasks_processing.go:69] worker 41 listening for tasks. I0329 02:55:07.115469 1 tasks_processing.go:69] worker 35 listening for tasks. I0329 02:55:07.115780 1 tasks_processing.go:71] worker 41 working on feature_gates task. I0329 02:55:07.115494 1 tasks_processing.go:71] worker 11 working on openshift_machine_api_events task. I0329 02:55:07.115820 1 tasks_processing.go:71] worker 54 working on machine_config_pools task. I0329 02:55:07.115858 1 tasks_processing.go:71] worker 31 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0329 02:55:07.115910 1 tasks_processing.go:71] worker 35 working on sap_datahubs task. I0329 02:55:07.115540 1 tasks_processing.go:71] worker 37 working on image_registries task. I0329 02:55:07.116030 1 tasks_processing.go:71] worker 56 working on machine_autoscalers task. I0329 02:55:07.116080 1 tasks_processing.go:71] worker 19 working on nodenetworkconfigurationpolicies task. I0329 02:55:07.115504 1 tasks_processing.go:69] worker 42 listening for tasks. I0329 02:55:07.116175 1 tasks_processing.go:71] worker 42 working on image task. I0329 02:55:07.115506 1 tasks_processing.go:71] worker 34 working on support_secret task. I0329 02:55:07.115511 1 tasks_processing.go:71] worker 40 working on aggregated_monitoring_cr_names task. I0329 02:55:07.116392 1 tasks_processing.go:71] worker 58 working on service_accounts task. I0329 02:55:07.115503 1 tasks_processing.go:71] worker 9 working on openstack_controlplanes task. I0329 02:55:07.115477 1 tasks_processing.go:69] worker 36 listening for tasks. I0329 02:55:07.115513 1 tasks_processing.go:71] worker 30 working on dvo_metrics task. I0329 02:55:07.116510 1 tasks_processing.go:71] worker 36 working on mutating_webhook_configurations task. I0329 02:55:07.115512 1 tasks_processing.go:69] worker 43 listening for tasks. I0329 02:55:07.115520 1 tasks_processing.go:71] worker 4 working on silenced_alerts task. I0329 02:55:07.115514 1 tasks_processing.go:71] worker 24 working on container_images task. I0329 02:55:07.115519 1 tasks_processing.go:69] worker 50 listening for tasks. I0329 02:55:07.115528 1 tasks_processing.go:69] worker 51 listening for tasks. I0329 02:55:07.115525 1 tasks_processing.go:71] worker 20 working on config_maps task. I0329 02:55:07.115527 1 tasks_processing.go:69] worker 63 listening for tasks. I0329 02:55:07.115531 1 tasks_processing.go:71] worker 5 working on clusterroles task. I0329 02:55:07.115520 1 tasks_processing.go:69] worker 44 listening for tasks. I0329 02:55:07.115535 1 tasks_processing.go:71] worker 2 working on networks task. I0329 02:55:07.115519 1 tasks_processing.go:71] worker 12 working on proxies task. I0329 02:55:07.116709 1 tasks_processing.go:71] worker 63 working on cluster_apiserver task. I0329 02:55:07.116713 1 tasks_processing.go:71] worker 51 working on storage_cluster task. I0329 02:55:07.115531 1 tasks_processing.go:69] worker 33 listening for tasks. I0329 02:55:07.116757 1 tasks_processing.go:71] worker 33 working on openstack_dataplanenodesets task. I0329 02:55:07.115535 1 tasks_processing.go:69] worker 52 listening for tasks. I0329 02:55:07.115609 1 tasks_processing.go:71] worker 46 working on crds task. I0329 02:55:07.115427 1 tasks_processing.go:69] worker 10 listening for tasks. I0329 02:55:07.115957 1 tasks_processing.go:71] worker 57 working on qemu_kubevirt_launcher_logs task. I0329 02:55:07.116860 1 tasks_processing.go:71] worker 50 working on olm_operators task. I0329 02:55:07.116951 1 tasks_processing.go:71] worker 10 working on storage_classes task. I0329 02:55:07.116970 1 tasks_processing.go:71] worker 52 working on install_plans task. I0329 02:55:07.115624 1 tasks_processing.go:71] worker 23 working on lokistack task. I0329 02:55:07.116267 1 tasks_processing.go:71] worker 61 working on machines task. I0329 02:55:07.115533 1 tasks_processing.go:71] worker 6 working on active_alerts task. I0329 02:55:07.116276 1 tasks_processing.go:71] worker 39 working on sap_config task. I0329 02:55:07.116284 1 tasks_processing.go:71] worker 55 working on image_pruners task. I0329 02:55:07.116289 1 tasks_processing.go:71] worker 13 working on oauths task. I0329 02:55:07.116700 1 tasks_processing.go:71] worker 44 working on operators_pods_and_events task. I0329 02:55:07.116705 1 tasks_processing.go:71] worker 43 working on openshift_logging task. I0329 02:55:07.116277 1 tasks_processing.go:71] worker 62 working on nodenetworkstates task. I0329 02:55:07.115513 1 tasks_processing.go:69] worker 49 listening for tasks. I0329 02:55:07.117891 1 tasks_processing.go:71] worker 49 working on pdbs task. I0329 02:55:07.118509 1 tasks_processing.go:71] worker 18 working on overlapping_namespace_uids task. E0329 02:55:07.118519 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0329 02:55:07.118529 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 2.981842ms to process 0 records I0329 02:55:07.118652 1 tasks_processing.go:74] worker 15 stopped. E0329 02:55:07.118664 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0329 02:55:07.118672 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 3.10973ms to process 0 records E0329 02:55:07.121409 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.137.117:9091: connect: connection refused I0329 02:55:07.121430 1 tasks_processing.go:74] worker 45 stopped. E0329 02:55:07.121441 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.137.117:9091: connect: connection refused I0329 02:55:07.121454 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 5.824934ms to process 0 records E0329 02:55:07.121412 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.99.13:9094: connect: connection refused I0329 02:55:07.121469 1 tasks_processing.go:74] worker 6 stopped. E0329 02:55:07.121462 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.99.13:9094: connect: connection refused E0329 02:55:07.121495 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp 172.30.99.13:9094: connect: connection refused I0329 02:55:07.121509 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 4.145698ms to process 0 records E0329 02:55:07.121502 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.137.117:9091: connect: connection refused E0329 02:55:07.121522 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp 172.30.99.13:9094: connect: connection refused I0329 02:55:07.121536 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 4.920867ms to process 0 records I0329 02:55:07.121583 1 tasks_processing.go:74] worker 4 stopped. I0329 02:55:07.121781 1 tasks_processing.go:74] worker 16 stopped. E0329 02:55:07.121549 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.137.117:9091: connect: connection refused I0329 02:55:07.121802 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.0857ms to process 0 records I0329 02:55:07.122890 1 tasks_processing.go:74] worker 35 stopped. I0329 02:55:07.122905 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 6.91336ms to process 0 records I0329 02:55:07.122963 1 tasks_processing.go:74] worker 3 stopped. I0329 02:55:07.122985 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 7.406701ms to process 0 records I0329 02:55:07.129082 1 tasks_processing.go:74] worker 28 stopped. I0329 02:55:07.129095 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 13.600484ms to process 0 records I0329 02:55:07.129105 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 13.009732ms to process 0 records I0329 02:55:07.129113 1 tasks_processing.go:74] worker 56 stopped. I0329 02:55:07.129361 1 tasks_processing.go:74] worker 60 stopped. I0329 02:55:07.129373 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0329 02:55:07.129377 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 13.802203ms to process 0 records I0329 02:55:07.129393 1 controller.go:326] The initial operator extension status is healthy I0329 02:55:07.129409 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0329 02:55:07.129417 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0329 02:55:07.129423 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0329 02:55:07.129443 1 controller.go:489] The operator is still being initialized I0329 02:55:07.129448 1 tasks_processing.go:74] worker 7 stopped. I0329 02:55:07.129459 1 controller.go:512] The operator is healthy I0329 02:55:07.129459 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 13.704555ms to process 0 records I0329 02:55:07.129472 1 tasks_processing.go:74] worker 26 stopped. I0329 02:55:07.129483 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 13.865829ms to process 0 records I0329 02:55:07.129495 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 13.928527ms to process 0 records I0329 02:55:07.129500 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 13.928207ms to process 0 records I0329 02:55:07.129504 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 13.37473ms to process 0 records I0329 02:55:07.129511 1 tasks_processing.go:74] worker 19 stopped. I0329 02:55:07.129516 1 tasks_processing.go:74] worker 53 stopped. I0329 02:55:07.129518 1 tasks_processing.go:74] worker 1 stopped. I0329 02:55:07.129549 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0329 02:55:07.129609 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0329 02:55:07.129693 1 operator.go:288] started I0329 02:55:07.129715 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0329 02:55:07.129717 1 tasks_processing.go:74] worker 41 stopped. I0329 02:55:07.129844 1 recorder.go:75] Recording config/featuregate with fingerprint=0da92dba33bb05a594832c0339a81dcb53280c2432c4b9309686799939984bc3 I0329 02:55:07.129857 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 13.927991ms to process 1 records I0329 02:55:07.129936 1 tasks_processing.go:74] worker 38 stopped. I0329 02:55:07.130007 1 recorder.go:75] Recording config/ingress with fingerprint=3f411d816b937ba0b4fae8d413054ceacbd06c6a5df1eddb6c1545850a5be164 I0329 02:55:07.130016 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 14.153444ms to process 1 records I0329 02:55:07.136084 1 tasks_processing.go:74] worker 11 stopped. I0329 02:55:07.136097 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 20.293503ms to process 0 records I0329 02:55:07.136108 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 19.363428ms to process 0 records I0329 02:55:07.136113 1 tasks_processing.go:74] worker 51 stopped. I0329 02:55:07.136258 1 tasks_processing.go:74] worker 48 stopped. I0329 02:55:07.136713 1 recorder.go:75] Recording config/infrastructure with fingerprint=d596264dd90b18c292f214c50ae829e95b8de551252c4588a0be6329b2c2597b I0329 02:55:07.136726 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 20.547367ms to process 1 records I0329 02:55:07.143466 1 tasks_processing.go:74] worker 42 stopped. I0329 02:55:07.143556 1 recorder.go:75] Recording config/image with fingerprint=35b8e72e10ccb889a327b857e1a950dab042906a8f2660b76fc3ea9f62c96461 I0329 02:55:07.143571 1 gather.go:177] gatherer "clusterconfig" function "image" took 27.250156ms to process 1 records I0329 02:55:07.143577 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 27.63741ms to process 0 records I0329 02:55:07.143599 1 tasks_processing.go:74] worker 54 stopped. I0329 02:55:07.143661 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=102e62ec74b42a8460d45a184e660e48c537adf8e258e661e97bc8b787cab57e I0329 02:55:07.143669 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 28.058882ms to process 1 records I0329 02:55:07.143677 1 tasks_processing.go:74] worker 21 stopped. I0329 02:55:07.144040 1 tasks_processing.go:74] worker 27 stopped. I0329 02:55:07.144315 1 recorder.go:75] Recording config/node/ip-10-0-0-214.ec2.internal with fingerprint=1290591f541ea4c9472b3fe43b84c1d21433b9965bfe737665e44c8e8b2548ea I0329 02:55:07.144371 1 recorder.go:75] Recording config/node/ip-10-0-1-191.ec2.internal with fingerprint=40017a78a5cf0e079dd2b6d0221f4aad8264578748fec7a12b22572f0235b429 I0329 02:55:07.144421 1 recorder.go:75] Recording config/node/ip-10-0-2-219.ec2.internal with fingerprint=252203ef4d67a8cd0785556388df135f2d952daa6d3cc401d297113d892bc961 I0329 02:55:07.144430 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 28.564806ms to process 3 records I0329 02:55:07.144436 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 26.621959ms to process 0 records I0329 02:55:07.144439 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 27.197035ms to process 0 records E0329 02:55:07.144444 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0329 02:55:07.144449 1 gather.go:177] gatherer "clusterconfig" function "machines" took 27.157792ms to process 0 records I0329 02:55:07.144456 1 tasks_processing.go:74] worker 61 stopped. I0329 02:55:07.144460 1 tasks_processing.go:74] worker 43 stopped. I0329 02:55:07.144464 1 tasks_processing.go:74] worker 23 stopped. I0329 02:55:07.144508 1 tasks_processing.go:74] worker 55 stopped. I0329 02:55:07.144689 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=a2a96f6cb31a204a850174aa8c00c3ac00f08c5c1f60a1c45f5ecc72193e9a31 I0329 02:55:07.144703 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 27.01204ms to process 1 records I0329 02:55:07.147382 1 tasks_processing.go:74] worker 8 stopped. I0329 02:55:07.147533 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=44a364b690c79268ef4e7d2b59d1a7a2da0a61f718f405586c47699014e7cfbe I0329 02:55:07.147597 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=6adad6a90e86584feaec3bc37efc06550269ba8c4b2b236eba5fc55b8424d589 I0329 02:55:07.147688 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=a5c217a704dbaf629c832a969f1d32c58d85f15338d77495eac9cf17553f6a7e I0329 02:55:07.147727 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=1fabbcbf47077a2f800564c614fc01f07aa08f190bfc859f8835766b731fbc83 I0329 02:55:07.147771 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=4dcd6fe837357be6d81fce35c975046f7badf146615fe7cf7e27c54e927b9c7e I0329 02:55:07.147810 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=f812ad2a8f78ec2916e3e3467a9da05a9a03cfb675dfb93a5808d129db6eeb26 I0329 02:55:07.147856 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=01eb6696778048c3c1fba0c43d9bf134107b3d5b43bfd48d9ce90c3b549998c8 I0329 02:55:07.147901 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=0b6924c5ff34ab4164141bc488c50a81b3cce6cf1b8ae35f82dcce4e3febd054 I0329 02:55:07.147856 1 gather_logs.go:145] no pods in namespace were found I0329 02:55:07.147941 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=f7f0629fa1a6942839414f749df6ffd5d3b570bf8f537302362eed577853e408 I0329 02:55:07.147987 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=1b0a5559b6610999e4280bcaee7d737d7a869c7ac1c6be128578f2bbfbf48c46 I0329 02:55:07.148020 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=df2f0d3ec3472ada5ad4f181a8026b5a50ef3bedbbdec24717fa155572d3cc21 I0329 02:55:07.148055 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=230196224f243c5821c6d6e7e91223db0f9de2aaced42be572ff8eda5003540f I0329 02:55:07.148088 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=ec80c0da5497706f03379c7b743d6e722040dcaf45b817e90f5373d645a231af I0329 02:55:07.148103 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 31.830586ms to process 13 records I0329 02:55:07.148114 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 29.963427ms to process 0 records I0329 02:55:07.148122 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 30.350273ms to process 0 records I0329 02:55:07.148128 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 31.049643ms to process 0 records I0329 02:55:07.148137 1 tasks_processing.go:74] worker 57 stopped. I0329 02:55:07.148149 1 tasks_processing.go:74] worker 62 stopped. I0329 02:55:07.148155 1 tasks_processing.go:74] worker 39 stopped. I0329 02:55:07.148717 1 tasks_processing.go:74] worker 17 stopped. I0329 02:55:07.148734 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 33.1183ms to process 0 records I0329 02:55:07.148896 1 tasks_processing.go:74] worker 22 stopped. I0329 02:55:07.148913 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 33.350673ms to process 0 records I0329 02:55:07.149486 1 tasks_processing.go:74] worker 12 stopped. I0329 02:55:07.149618 1 recorder.go:75] Recording config/proxy with fingerprint=71b725efe786ab8172f55726b628e745c286ae147011c73afca29c72213bd5ad I0329 02:55:07.149657 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 32.770443ms to process 1 records I0329 02:55:07.149818 1 tasks_processing.go:74] worker 9 stopped. I0329 02:55:07.149838 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 33.366031ms to process 0 records I0329 02:55:07.149869 1 tasks_processing.go:74] worker 33 stopped. I0329 02:55:07.149879 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 33.104507ms to process 0 records I0329 02:55:07.150474 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0329 02:55:07.150501 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0329 02:55:07.150534 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0329 02:55:07.150541 1 controller.go:212] Source scaController *sca.Controller is not ready I0329 02:55:07.150547 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0329 02:55:07.150612 1 controller.go:489] The operator is still being initialized I0329 02:55:07.150624 1 controller.go:512] The operator is healthy I0329 02:55:07.150664 1 tasks_processing.go:74] worker 2 stopped. I0329 02:55:07.150952 1 recorder.go:75] Recording config/network with fingerprint=acad183a827bea245b3434ff5c535d8269d182bef95285988ed69fee88d3f84b I0329 02:55:07.151007 1 gather.go:177] gatherer "clusterconfig" function "networks" took 33.977771ms to process 1 records I0329 02:55:07.151220 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=51957db52dc5cfe324530bb033840b80d58ade78b45d0f25887c050c1842864a I0329 02:55:07.151305 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=e09cb088fa668775a1007299d8f9b568713997be15223155f6a9976eca3c471c I0329 02:55:07.151324 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 33.737956ms to process 2 records E0329 02:55:07.151350 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0329 02:55:07.151364 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 34.40038ms to process 0 records I0329 02:55:07.151426 1 tasks_processing.go:74] worker 10 stopped. I0329 02:55:07.151496 1 tasks_processing.go:74] worker 37 stopped. I0329 02:55:07.151536 1 tasks_processing.go:74] worker 34 stopped. W0329 02:55:07.151958 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0329 02:55:07.152753 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=e7da31557b5ebbe5aefa83f86e3d8901508cfd8ac8fd02fead1f633edd9685fe I0329 02:55:07.152774 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 34.764627ms to process 1 records I0329 02:55:07.152901 1 tasks_processing.go:74] worker 47 stopped. I0329 02:55:07.152918 1 recorder.go:75] Recording config/authentication with fingerprint=50f375e8bedaea54ca021865ae3b211548d2142877ec9173d7e884dc881419a6 I0329 02:55:07.152925 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 35.474604ms to process 1 records I0329 02:55:07.152935 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 36.043482ms to process 0 records I0329 02:55:07.152972 1 tasks_processing.go:74] worker 32 stopped. I0329 02:55:07.153064 1 recorder.go:75] Recording config/apiserver with fingerprint=e211d5e266e8ffe0221caeb32a88e880360c56c6869d832c16a43ffd66bca54e I0329 02:55:07.153090 1 tasks_processing.go:74] worker 63 stopped. I0329 02:55:07.153092 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 34.967983ms to process 1 records I0329 02:55:07.153251 1 tasks_processing.go:74] worker 14 stopped. I0329 02:55:07.153411 1 recorder.go:75] Recording config/version with fingerprint=d3334a395ebfad170dcdf88baea03008296e5058b56e441ee35bab875f70b441 I0329 02:55:07.153425 1 recorder.go:75] Recording config/id with fingerprint=72cfb86682dfaaaac2fafcf08434f3a1327745880014e646cfc613019f04f14f I0329 02:55:07.153431 1 gather.go:177] gatherer "clusterconfig" function "version" took 37.294034ms to process 2 records I0329 02:55:07.153513 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=8696d3c9342684054b53e802997941ef4950423ad9a305be9414942b275d52f6 I0329 02:55:07.153524 1 tasks_processing.go:74] worker 49 stopped. I0329 02:55:07.153538 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=1fc691807779d37ee307dd332a3feb01fef71090b6e352d2d9b50d8539b3ad0f I0329 02:55:07.153560 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=a2c51fb3bffd98aa59b94ae33c7ea89b53764eff77caefc28df566158aafdaf1 I0329 02:55:07.153584 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=79848e90694e0cf2bf9f3778b396b60d137bc95934f5fcf984fe174f9b26bac4 I0329 02:55:07.153604 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=86a243624f5908747ef5c0c40dff1f2111dd124e968968afa9af182695ade6ea I0329 02:55:07.153620 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=6ea9fd62e7c672501e827a7fa20bb2584feac2ae84f9069900e876061de1845e I0329 02:55:07.153633 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=07957af3cac51b53286edd7e0e2668511cfa041fc20ce461e1962758e29dd948 I0329 02:55:07.153650 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=8947b2b7747909e1f29146d2d492df9050ad2bb227ff6855f668301b239953ca I0329 02:55:07.153667 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=5ccb8bd882468831cb9a64d917a27ecc15a5d33b61473bd531a25a745f27c8fa I0329 02:55:07.153677 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 35.399261ms to process 9 records I0329 02:55:07.153797 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=3ba5d9f27483b04efdad33d367868763f1f676ad35b7b223728195454a00666a I0329 02:55:07.153817 1 tasks_processing.go:74] worker 36 stopped. I0329 02:55:07.153837 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=0c7e5669d411639d45a7dbe74710af755e4105707964719dcb7dcc5f2d24c5a6 I0329 02:55:07.153866 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=0a168c96eff2b2c16f6e8c7852ab1e15ecc96a774a67e0b74e7b85cdd63cbbff I0329 02:55:07.153877 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 36.983017ms to process 3 records I0329 02:55:07.153954 1 tasks_processing.go:74] worker 13 stopped. I0329 02:55:07.154176 1 recorder.go:75] Recording config/oauth with fingerprint=ea2fec4e0929bc4f9435385afb67dc47b1c46bbef0d8e5fd9e3983e7748d3e49 I0329 02:55:07.154191 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 36.052796ms to process 1 records I0329 02:55:07.154201 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 38.126089ms to process 0 records I0329 02:55:07.154208 1 tasks_processing.go:74] worker 59 stopped. I0329 02:55:07.158039 1 tasks_processing.go:74] worker 18 stopped. I0329 02:55:07.158130 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0329 02:55:07.158167 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 39.512385ms to process 1 records I0329 02:55:07.161816 1 tasks_processing.go:74] worker 25 stopped. I0329 02:55:07.161858 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0329 02:55:07.161872 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 46.188534ms to process 1 records I0329 02:55:07.162330 1 tasks_processing.go:74] worker 46 stopped. I0329 02:55:07.162869 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=cd80da4f92ef0a8d3d9324e1f85eba2ce3eb5c9aa9fb78670d42ccb64a57565b I0329 02:55:07.163063 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=64bcc111b8749f2adb69a7343f4916fe414b09d3fa0d76360f67be05dffa2023 I0329 02:55:07.163073 1 gather.go:177] gatherer "clusterconfig" function "crds" took 45.50874ms to process 2 records I0329 02:55:07.170269 1 tasks_processing.go:74] worker 5 stopped. I0329 02:55:07.170659 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=51729cdad78630b93c1824ce5c958a1ba3e15710f7cf2f14001c84242fda8cd5 I0329 02:55:07.170812 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=6111baa554d9fe80694bed5d42a65e0274017161d71a789b5bc3271fa39e6d1c I0329 02:55:07.170841 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 53.603279ms to process 2 records I0329 02:55:07.170938 1 recorder.go:75] Recording config/olm_operators with fingerprint=2fa8d3652878185a287fa8a0e688f1b342eb4cf074d04fa49dd2ac83e474b4a5 I0329 02:55:07.170952 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 53.733569ms to process 1 records I0329 02:55:07.170970 1 tasks_processing.go:74] worker 50 stopped. I0329 02:55:07.172911 1 tasks_processing.go:74] worker 40 stopped. I0329 02:55:07.172928 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 56.523443ms to process 0 records I0329 02:55:07.174677 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0329 02:55:07.177627 1 tasks_processing.go:74] worker 24 stopped. I0329 02:55:07.178930 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-cqg49 with fingerprint=cc11c6fae8a16e70443633ceb9e502907aab330f6f1b4372e146c267887c3aae I0329 02:55:07.179030 1 recorder.go:75] Recording config/running_containers with fingerprint=ba1f50cfb2e1740b7835bd99c1bb9d81eec6432bad7faeb033714004063eaa57 I0329 02:55:07.179043 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 61.044702ms to process 2 records I0329 02:55:07.179499 1 tasks_processing.go:74] worker 31 stopped. I0329 02:55:07.179513 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 63.622566ms to process 0 records I0329 02:55:07.184764 1 tasks_processing.go:74] worker 20 stopped. E0329 02:55:07.184801 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0329 02:55:07.184815 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0329 02:55:07.184819 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0329 02:55:07.184830 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0329 02:55:07.184862 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0329 02:55:07.184871 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0329 02:55:07.184921 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0329 02:55:07.184973 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0329 02:55:07.185095 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0329 02:55:07.185120 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0329 02:55:07.185128 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 68.134984ms to process 7 records I0329 02:55:07.188490 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0329 02:55:07.188504 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0329 02:55:07.188515 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0329 02:55:07.192678 1 base_controller.go:82] Caches are synced for ConfigController I0329 02:55:07.192688 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0329 02:55:07.196271 1 configmapobserver.go:84] configmaps "insights-config" not found I0329 02:55:07.230514 1 base_controller.go:82] Caches are synced for LoggingSyncer I0329 02:55:07.230528 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0329 02:55:07.261536 1 tasks_processing.go:74] worker 0 stopped. E0329 02:55:07.261557 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0329 02:55:07.261566 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2pb44o1k46ho5pnrk0fefudvc264u4d0-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2pb44o1k46ho5pnrk0fefudvc264u4d0-primary-cert-bundle-secret" not found I0329 02:55:07.261642 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=4ba8ee0ea048b14424ce0826327561b646e1b4a413c7ac90873d9d2f7570a0e8 I0329 02:55:07.261661 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 145.995987ms to process 1 records I0329 02:55:07.270847 1 gather_cluster_operator_pods_and_events.go:121] Found 12 pods with 15 containers I0329 02:55:07.270860 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1677721 bytes I0329 02:55:07.271847 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-qstlx pod in namespace openshift-dns (previous: false). I0329 02:55:07.274993 1 cluster_transfer.go:102] no available accepted cluster transfer I0329 02:55:07.275009 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0329 02:55:07.346557 1 sca.go:266] etc-pki-entitlement secret successfully created I0329 02:55:07.346577 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0329 02:55:07.374065 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-qstlx pod in namespace openshift-dns (previous: false). I0329 02:55:07.572891 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-v6s7m pod in namespace openshift-dns (previous: false). I0329 02:55:07.772061 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-v6s7m pod in namespace openshift-dns (previous: false). I0329 02:55:07.974619 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-zf2wq pod in namespace openshift-dns (previous: false). W0329 02:55:08.151512 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0329 02:55:08.171336 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-zf2wq pod in namespace openshift-dns (previous: false). I0329 02:55:08.372699 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8x6wk pod in namespace openshift-dns (previous: false). I0329 02:55:08.572465 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-jznc7 pod in namespace openshift-dns (previous: false). I0329 02:55:08.772672 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-sw7sn pod in namespace openshift-dns (previous: false). I0329 02:55:08.971438 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-59675d655f-gqmc9 pod in namespace openshift-ingress (previous: false). W0329 02:55:09.150957 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0329 02:55:09.174421 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-59675d655f-vzmnq pod in namespace openshift-ingress (previous: false). I0329 02:55:09.373512 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-94b485d95-ntjtk pod in namespace openshift-ingress (previous: false). I0329 02:55:09.573180 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-6zhv7 pod in namespace openshift-ingress-canary (previous: false). I0329 02:55:09.582287 1 tasks_processing.go:74] worker 29 stopped. I0329 02:55:09.582400 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=9b156a1cd5f01cfe9222c5011cc6456428d0028a56fd50705ac6785cf3385006 I0329 02:55:09.582448 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0329 02:55:09.582473 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=f247e054f0c982755f0c3efe2c1e3301e0087dfdcd67623497bd8be5bbe8c3af I0329 02:55:09.582484 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0329 02:55:09.582506 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=71e752323fe0577eccc7ee6c4598e8fea55d3a7e90afba57821639fd7948dec4 I0329 02:55:09.582523 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0329 02:55:09.582545 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=46d2d87b2f554c32007675d1871c99d895645a9abfb80a3b90a2e3f0cccf2832 I0329 02:55:09.582572 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=20d834a31af3cc5723186054f049c32831fc8875d86ff32dc769988e3f6aecca I0329 02:55:09.582596 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=3ab253567e8b7cf5f0834090b08d9146fca60f75a09425695a32f039554145d0 I0329 02:55:09.582625 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=0bb8021c06940c969faf942f06a533dea7818e0423039563c797937c9b92ad29 I0329 02:55:09.582885 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=7e5ac431528f9a685916dace4b86a10b8f284888660878c527409757b9f69e96 I0329 02:55:09.582904 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=5c118b8a74636d2d1628bdf7d7d2c2a2556587bc07d358082ecf64b26a52afae I0329 02:55:09.582914 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0329 02:55:09.582929 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=c9d811ef7c69293edb94dfabca86227e0d88f3f33f8e7e3602889fb003f03927 I0329 02:55:09.582939 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0329 02:55:09.582958 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=e74cb48b3d53ed657b51dcf566aebd55b491039aad2f7e31acd075f83dc0e8d3 I0329 02:55:09.582967 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0329 02:55:09.582987 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=1c62f7c006474a53a746a897622ba369d48d61b1e2d271576a36164b2526b8da I0329 02:55:09.582997 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0329 02:55:09.583017 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=cde2958b8502ccad07613e6326fdf30ae6f2ffdf7c23482027510cd081443eae I0329 02:55:09.583117 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=c8d2eb522121e4a2237a1cc435042a6301bb0a14a5a2372c1149e248959d08eb I0329 02:55:09.583126 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0329 02:55:09.583147 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0329 02:55:09.583171 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0329 02:55:09.583191 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=d0ddddff3f0a34f15e9c7b03cabd51a4ebbc3a3083170487799294c1a166c00c I0329 02:55:09.583215 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=faa99e8894a3461d62bdf219265151219f7015ca969fb0f7193a0bcaabc5dc2e I0329 02:55:09.583225 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0329 02:55:09.583254 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=87e2d8d4aee26ba75c507aab598a26b4db67f13c74470cb1df68f5a39d8b5039 I0329 02:55:09.583262 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0329 02:55:09.583278 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=d51da9c8f376655602d839be4ed0d48671a45f9f80081972aa066e2a28f43733 I0329 02:55:09.583287 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0329 02:55:09.583300 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=d7aed1606dc7bf363d0f79378b6fa6a2d8e334150bd30b07e46b34ce30db1941 I0329 02:55:09.583315 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=c8dc8e10d71c34724d2b8d0bcda857ac452cf5c466ffae6db5a6ea9084e6664b I0329 02:55:09.583331 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=7e67d8fbdac716d92f20ada85371ca59cca10d450b540ff28b53497594532fae I0329 02:55:09.583355 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=913c57a73770c47ad4c1719ce3350cd6f2a26eb07dbe4a7ee1d031113bdf16d5 I0329 02:55:09.583364 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0329 02:55:09.583386 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=2dc88b02879d08c9db1214549f8085dd488ace018ac811a375ca2d68f5f9fcf2 I0329 02:55:09.583403 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0329 02:55:09.583411 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0329 02:55:09.583417 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.466641925s to process 39 records I0329 02:55:09.771961 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-7ch5s pod in namespace openshift-ingress-canary (previous: false). I0329 02:55:09.971122 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-gqmz9 pod in namespace openshift-ingress-canary (previous: false). W0329 02:55:10.150724 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0329 02:55:10.173161 1 tasks_processing.go:74] worker 44 stopped. I0329 02:55:10.173334 1 recorder.go:75] Recording events/openshift-dns with fingerprint=0b146fa1a99f5bf3666105ccfb107b0988b6cbdaa5a908e895f856fab9290406 I0329 02:55:10.173369 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=7bcb2a8966f59902d0705682102153915ae51e99f4560b7fdfa961cc3a75b109 I0329 02:55:10.173437 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=9a949b8c414076076d715ab0371560b4f1102ee37a58571bccb3fb976d47b197 I0329 02:55:10.173478 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=a8093c27c4d5be1e74ff0170965ea1c4265d69c22bd78254960392c4b36c6f61 I0329 02:55:10.173487 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-qstlx/dns_current.log with fingerprint=c8f5986d7ac2cf186e1894589807dc8fa51209cea3d2e68b2bc66f3f97a70fd0 I0329 02:55:10.173494 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-qstlx/kube-rbac-proxy_current.log with fingerprint=db5dbe73d8d5a15317e1f99ff37fb8b9a654b0fc1699ffebf1dc61999a284f53 I0329 02:55:10.173500 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-v6s7m/dns_current.log with fingerprint=d76ef4ef6fb9b92bd50c82a8a61efa31815ba1ba3b514966262f3d0e30b42dea I0329 02:55:10.173507 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-v6s7m/kube-rbac-proxy_current.log with fingerprint=1356679e0cd500dfda06b4e9d0a0a9dd014efd7ea649c4417080cd6d0ff27f24 I0329 02:55:10.173512 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-zf2wq/dns_current.log with fingerprint=780cb9841c65744418546042e9a4ebd884804b3bef53e8d9b518a2da34ba114b I0329 02:55:10.173519 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-zf2wq/kube-rbac-proxy_current.log with fingerprint=5a42cbc6d07b2d98d1763c1fa67a344c3b1ce15ae03122638c5bf5ab44bc8c35 I0329 02:55:10.173524 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-8x6wk/dns-node-resolver_current.log with fingerprint=b01adcbc0949767dacf6db7269ed9b5505bf2b90a43ff9481f6a534332ed634b I0329 02:55:10.173529 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-jznc7/dns-node-resolver_current.log with fingerprint=1686e058b1407bb416b1b3dd22c0fc7b8871de33fddc55619f4c3886c8aa762f I0329 02:55:10.173533 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-sw7sn/dns-node-resolver_current.log with fingerprint=d6ca13b0ec16f0ab853cccb42ae43489c792f2f72c80b6fc7226dcf597b39e44 I0329 02:55:10.173589 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-59675d655f-gqmc9/router_current.log with fingerprint=6a9194fae3b661de4dc43a79d82e6895dec63b905446d7821b22e98f97877b21 I0329 02:55:10.173621 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-59675d655f-vzmnq/router_current.log with fingerprint=9ee0a30368f51982276aaad044dbb8aaf5cfa253ff589e4aaba10fdba81c5944 I0329 02:55:10.173654 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-94b485d95-ntjtk/router_current.log with fingerprint=c3ee4d9a2e7630cec749f29d834f8c328f75b8da6f70865b0586d18321c498f2 I0329 02:55:10.173660 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-6zhv7/serve-healthcheck-canary_current.log with fingerprint=375bb7fbf6f4c1950e404f8897a71f17032c6246750a2ecc37332afe8ca7b4c3 I0329 02:55:10.173664 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-7ch5s/serve-healthcheck-canary_current.log with fingerprint=fb289be5cfbb887117ada45ec6aa94abfc79acdccc0264d6d3c774251f8c05cd I0329 02:55:10.173667 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-gqmz9/serve-healthcheck-canary_current.log with fingerprint=e228c05b09d7c2affacf2bcb0dd974fb53b277ca723348c947fd0dfc15797067 I0329 02:55:10.173673 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 3.055605815s to process 19 records W0329 02:55:11.151480 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0329 02:55:12.149081 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0329 02:55:12.149111 1 tasks_processing.go:74] worker 30 stopped. E0329 02:55:12.149126 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0329 02:55:12.149151 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0329 02:55:12.149162 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.032589597s to process 1 records I0329 02:55:19.961961 1 tasks_processing.go:74] worker 52 stopped. I0329 02:55:19.962005 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0329 02:55:19.962016 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.844961527s to process 1 records I0329 02:55:20.722725 1 tasks_processing.go:74] worker 58 stopped. I0329 02:55:20.723043 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=4aff5860ec073ddfdf1bec96303c96ef7753a0d476f8315de0a2062bde3a2fd5 I0329 02:55:20.723060 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.606309007s to process 1 records E0329 02:55:20.723124 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.607s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"tsdb_status\" failed with an error, function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"metrics\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0329 02:55:20.724249 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "tsdb_status" failed with an error, function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "metrics" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0329 02:55:20.724265 1 periodic.go:209] Running workloads gatherer I0329 02:55:20.724283 1 tasks_processing.go:45] number of workers: 2 I0329 02:55:20.724293 1 tasks_processing.go:69] worker 1 listening for tasks. I0329 02:55:20.724297 1 tasks_processing.go:71] worker 1 working on workload_info task. I0329 02:55:20.724306 1 tasks_processing.go:69] worker 0 listening for tasks. I0329 02:55:20.724325 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0329 02:55:20.730752 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-219.ec2.internal... I0329 02:55:20.731035 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-214.ec2.internal... I0329 02:55:20.731045 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-191.ec2.internal... I0329 02:55:20.746389 1 tasks_processing.go:74] worker 0 stopped. I0329 02:55:20.746406 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 22.043424ms to process 0 records I0329 02:55:24.695717 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0329 02:55:24.720180 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0329 02:55:24.727638 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0329 02:55:24.735655 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0329 02:55:24.743031 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (7ms) I0329 02:55:24.749985 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (7ms) I0329 02:55:24.757298 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (7ms) I0329 02:55:24.764871 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (8ms) I0329 02:55:24.772092 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (7ms) I0329 02:55:24.779181 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (7ms) I0329 02:55:24.786738 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (8ms) I0329 02:55:24.793737 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (7ms) I0329 02:55:24.827996 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (34ms) I0329 02:55:24.927334 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (99ms) I0329 02:55:25.028966 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (102ms) I0329 02:55:25.128255 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (99ms) I0329 02:55:25.228218 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0329 02:55:25.327399 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (99ms) I0329 02:55:25.428039 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (101ms) I0329 02:55:25.528884 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (101ms) I0329 02:55:25.628341 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (99ms) I0329 02:55:25.727874 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0329 02:55:25.828508 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (101ms) I0329 02:55:25.927837 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (99ms) I0329 02:55:26.027480 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0329 02:55:26.127654 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (100ms) I0329 02:55:26.230191 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (103ms) I0329 02:55:26.328369 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (98ms) I0329 02:55:26.427381 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (99ms) I0329 02:55:26.528097 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (101ms) I0329 02:55:26.629929 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (102ms) I0329 02:55:26.727491 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (98ms) I0329 02:55:26.829137 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (102ms) I0329 02:55:26.927450 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (98ms) I0329 02:55:27.029893 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (102ms) I0329 02:55:27.128081 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (98ms) I0329 02:55:27.228074 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0329 02:55:27.327586 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (99ms) I0329 02:55:27.429659 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (102ms) I0329 02:55:27.528349 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (99ms) I0329 02:55:27.633495 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (105ms) I0329 02:55:27.733160 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (100ms) I0329 02:55:27.840050 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (107ms) I0329 02:55:27.927782 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (88ms) I0329 02:55:28.028680 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0329 02:55:28.127986 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (99ms) I0329 02:55:28.230421 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (102ms) I0329 02:55:28.329384 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (99ms) I0329 02:55:28.428848 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (99ms) I0329 02:55:28.428882 1 tasks_processing.go:74] worker 1 stopped. E0329 02:55:28.428893 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: received unexpected status code 500 Internal Server Error from https://10.129.0.17:8443/gather_runtime_info I0329 02:55:28.429201 1 recorder.go:75] Recording config/workload_info with fingerprint=c3255425b74671cfd420e685028d6cc8bacd88717130d1e75f71f9ce2b912381 I0329 02:55:28.429216 1 gather.go:177] gatherer "workloads" function "workload_info" took 7.704577787s to process 1 records E0329 02:55:28.429259 1 periodic.go:247] "Unhandled Error" err="workloads failed after 7.704s with: function \"workload_info\" failed with an error" I0329 02:55:28.430362 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0329 02:55:28.430374 1 periodic.go:209] Running conditional gatherer I0329 02:55:28.435754 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0329 02:55:29.610606 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0329 02:55:29.613268 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp 172.30.137.117:9091: connect: connection refused I0329 02:55:29.613340 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0329 02:55:29.619023 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0329 02:55:29.619039 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619044 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619047 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619050 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619053 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619057 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619062 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619066 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619071 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0329 02:55:29.619075 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0329 02:55:29.619088 1 tasks_processing.go:45] number of workers: 3 I0329 02:55:29.619099 1 tasks_processing.go:69] worker 2 listening for tasks. I0329 02:55:29.619103 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0329 02:55:29.619110 1 tasks_processing.go:69] worker 0 listening for tasks. I0329 02:55:29.619122 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0329 02:55:29.619127 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0329 02:55:29.619129 1 tasks_processing.go:69] worker 1 listening for tasks. I0329 02:55:29.619142 1 tasks_processing.go:74] worker 1 stopped. I0329 02:55:29.619194 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0329 02:55:29.619208 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 830ns to process 1 records I0329 02:55:29.619250 1 tasks_processing.go:74] worker 0 stopped. I0329 02:55:29.619271 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0329 02:55:29.619278 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.16µs to process 1 records I0329 02:55:29.619384 1 tasks_processing.go:74] worker 2 stopped. I0329 02:55:29.619396 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 251.84µs to process 0 records I0329 02:55:29.619412 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0329 02:55:29.619427 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0329 02:55:29.646638 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=c844ffc2e3a52ada8324cb3da774097d0ddbff0c27cee6e6cd8212c9350a2de1 I0329 02:55:29.646774 1 diskrecorder.go:70] Writing 126 records to /var/lib/insights-operator/insights-2026-03-29-025529.tar.gz I0329 02:55:29.655590 1 diskrecorder.go:51] Wrote 126 records to disk in 8ms I0329 02:55:29.655630 1 periodic.go:278] Gathering cluster info every 2h0m0s I0329 02:55:29.655650 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0329 02:56:37.116225 1 diskrecorder.go:223] Found files to send: insights-2026-03-29-025529.tar.gz I0329 02:56:37.116270 1 insightsuploader.go:150] Checking archives to upload periodically every 15m53.389557555s I0329 02:56:37.116279 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0329 02:56:37.126913 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0329 02:56:37.344827 1 requests.go:87] Successfully reported id=2026-03-29T02:56:37Z x-rh-insights-request-id=17a06900740b41538db86d6d09d28cf1, wrote=55461 I0329 02:56:37.344887 1 insightsuploader.go:187] Uploaded report successfully in 228.599272ms I0329 02:56:37.344910 1 controller.go:128] Initializing last reported time to 2026-03-29T02:56:37Z I0329 02:56:37.344938 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0329 02:56:37.344946 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0329 02:56:37.344954 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0329 02:56:37.349853 1 controller.go:512] The operator is healthy I0329 02:57:07.135644 1 controller.go:512] The operator is healthy I0329 02:57:41.066874 1 insightsreport.go:137] Pulling report from smart-proxy I0329 02:57:41.066905 1 insightsreport.go:149] Retrieving report I0329 02:57:41.073794 1 requests.go:110] Retrieving report for cluster: 6184997e-d53c-4ca4-9e17-2917a58d855e I0329 02:57:41.073809 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/6184997e-d53c-4ca4-9e17-2917a58d855e/reports I0329 02:57:41.076501 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/6184997e-d53c-4ca4-9e17-2917a58d855e/reports I0329 02:57:41.312047 1 insightsreport.go:184] Report retrieved I0329 02:57:41.339570 1 insightsreport.go:239] Report retrieved correctly I0329 02:59:07.136393 1 controller.go:512] The operator is healthy I0329 03:00:07.108851 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0329 03:00:07.114025 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0329 03:00:07.114046 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0329 03:00:07.117522 1 secretconfigobserver.go:119] support secret does not exist I0329 03:01:07.136550 1 controller.go:512] The operator is healthy I0329 03:03:07.134416 1 controller.go:512] The operator is healthy I0329 03:05:07.107430 1 configmapobserver.go:84] configmaps "insights-config" not found I0329 03:05:07.118517 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0329 03:05:07.123095 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0329 03:05:07.123112 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0329 03:05:07.126709 1 secretconfigobserver.go:119] support secret does not exist I0329 03:05:07.134658 1 controller.go:512] The operator is healthy I0329 03:07:07.135963 1 controller.go:512] The operator is healthy I0329 03:09:07.135876 1 controller.go:512] The operator is healthy