I0505 13:05:41.030147 1 cmd.go:253] Using service-serving-cert provided certificates I0505 13:05:41.030604 1 observer_polling.go:159] Starting file observer I0505 13:05:41.703959 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0505 13:05:41.704247 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0505 13:05:41.705191 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0505 13:05:41.705222 1 secure_serving.go:57] Forcing use of http/1.1 only W0505 13:05:41.705243 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0505 13:05:41.705249 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0505 13:05:41.705256 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0505 13:05:41.705260 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0505 13:05:41.705263 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0505 13:05:41.705267 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0505 13:05:41.710195 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0505 13:05:41.710270 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"2f5e888e-5b28-4d59-a27b-699af3bd236e", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0505 13:05:41.711221 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0505 13:05:41.711244 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0505 13:05:41.711230 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0505 13:05:41.711231 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0505 13:05:41.711261 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0505 13:05:41.711267 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0505 13:05:41.711486 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0505 13:05:41.711712 1 secure_serving.go:213] Serving securely on [::]:8443 I0505 13:05:41.711739 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0505 13:05:41.718957 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0505 13:05:41.718980 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0505 13:05:41.719102 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0505 13:05:41.724692 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:05:41.724709 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0505 13:05:41.731557 1 secretconfigobserver.go:119] support secret does not exist I0505 13:05:41.737299 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:05:41.743462 1 secretconfigobserver.go:119] support secret does not exist I0505 13:05:41.745028 1 recorder.go:161] Pruning old reports every 7h15m6s, max age is 288h0m0s I0505 13:05:41.751809 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0505 13:05:41.751833 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0505 13:05:41.751842 1 periodic.go:209] Running clusterconfig gatherer I0505 13:05:41.751847 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0505 13:05:41.751872 1 insightsreport.go:296] Starting report retriever I0505 13:05:41.751886 1 tasks_processing.go:45] number of workers: 64 I0505 13:05:41.751884 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0505 13:05:41.751906 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 13:05:41.751920 1 tasks_processing.go:69] worker 9 listening for tasks. I0505 13:05:41.751928 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 13:05:41.751936 1 tasks_processing.go:69] worker 2 listening for tasks. I0505 13:05:41.751943 1 tasks_processing.go:69] worker 3 listening for tasks. I0505 13:05:41.751956 1 tasks_processing.go:69] worker 4 listening for tasks. I0505 13:05:41.751967 1 tasks_processing.go:69] worker 5 listening for tasks. I0505 13:05:41.751969 1 tasks_processing.go:71] worker 4 working on machines task. I0505 13:05:41.751972 1 tasks_processing.go:71] worker 5 working on nodenetworkstates task. I0505 13:05:41.751976 1 tasks_processing.go:69] worker 32 listening for tasks. I0505 13:05:41.751987 1 tasks_processing.go:71] worker 32 working on image_registries task. I0505 13:05:41.751989 1 tasks_processing.go:69] worker 21 listening for tasks. I0505 13:05:41.751994 1 tasks_processing.go:69] worker 10 listening for tasks. I0505 13:05:41.752003 1 tasks_processing.go:71] worker 10 working on container_images task. I0505 13:05:41.752003 1 tasks_processing.go:69] worker 50 listening for tasks. I0505 13:05:41.752007 1 tasks_processing.go:69] worker 22 listening for tasks. I0505 13:05:41.752020 1 tasks_processing.go:69] worker 23 listening for tasks. I0505 13:05:41.752021 1 tasks_processing.go:69] worker 42 listening for tasks. I0505 13:05:41.752020 1 tasks_processing.go:69] worker 41 listening for tasks. I0505 13:05:41.752032 1 tasks_processing.go:69] worker 33 listening for tasks. I0505 13:05:41.752032 1 tasks_processing.go:69] worker 44 listening for tasks. I0505 13:05:41.752026 1 tasks_processing.go:69] worker 43 listening for tasks. I0505 13:05:41.752040 1 tasks_processing.go:69] worker 45 listening for tasks. I0505 13:05:41.752041 1 tasks_processing.go:69] worker 28 listening for tasks. I0505 13:05:41.752045 1 tasks_processing.go:69] worker 46 listening for tasks. I0505 13:05:41.752040 1 tasks_processing.go:69] worker 27 listening for tasks. I0505 13:05:41.752052 1 tasks_processing.go:69] worker 31 listening for tasks. I0505 13:05:41.752055 1 tasks_processing.go:69] worker 24 listening for tasks. I0505 13:05:41.752057 1 tasks_processing.go:69] worker 48 listening for tasks. I0505 13:05:41.752064 1 tasks_processing.go:69] worker 35 listening for tasks. I0505 13:05:41.752059 1 tasks_processing.go:69] worker 30 listening for tasks. I0505 13:05:41.752052 1 tasks_processing.go:69] worker 47 listening for tasks. I0505 13:05:41.752092 1 tasks_processing.go:69] worker 39 listening for tasks. I0505 13:05:41.752094 1 tasks_processing.go:69] worker 26 listening for tasks. I0505 13:05:41.752095 1 tasks_processing.go:69] worker 6 listening for tasks. I0505 13:05:41.752091 1 tasks_processing.go:69] worker 15 listening for tasks. I0505 13:05:41.752091 1 tasks_processing.go:69] worker 36 listening for tasks. I0505 13:05:41.752103 1 tasks_processing.go:69] worker 29 listening for tasks. I0505 13:05:41.752106 1 tasks_processing.go:69] worker 12 listening for tasks. I0505 13:05:41.752104 1 tasks_processing.go:69] worker 8 listening for tasks. I0505 13:05:41.752107 1 tasks_processing.go:69] worker 11 listening for tasks. I0505 13:05:41.752114 1 tasks_processing.go:71] worker 1 working on sap_pods task. I0505 13:05:41.752114 1 tasks_processing.go:69] worker 19 listening for tasks. I0505 13:05:41.752119 1 tasks_processing.go:69] worker 14 listening for tasks. I0505 13:05:41.752120 1 tasks_processing.go:69] worker 16 listening for tasks. I0505 13:05:41.752123 1 tasks_processing.go:69] worker 51 listening for tasks. I0505 13:05:41.752126 1 tasks_processing.go:69] worker 52 listening for tasks. I0505 13:05:41.752126 1 tasks_processing.go:69] worker 55 listening for tasks. I0505 13:05:41.752123 1 tasks_processing.go:71] worker 0 working on machine_autoscalers task. I0505 13:05:41.752132 1 tasks_processing.go:69] worker 58 listening for tasks. I0505 13:05:41.752131 1 tasks_processing.go:69] worker 17 listening for tasks. I0505 13:05:41.752135 1 tasks_processing.go:69] worker 63 listening for tasks. I0505 13:05:41.752135 1 tasks_processing.go:71] worker 2 working on metrics task. I0505 13:05:41.752138 1 tasks_processing.go:69] worker 60 listening for tasks. I0505 13:05:41.752071 1 tasks_processing.go:69] worker 25 listening for tasks. I0505 13:05:41.752059 1 tasks_processing.go:69] worker 34 listening for tasks. I0505 13:05:41.752100 1 tasks_processing.go:69] worker 40 listening for tasks. I0505 13:05:41.752100 1 tasks_processing.go:69] worker 37 listening for tasks. I0505 13:05:41.752151 1 tasks_processing.go:69] worker 61 listening for tasks. I0505 13:05:41.752061 1 tasks_processing.go:69] worker 49 listening for tasks. I0505 13:05:41.752112 1 tasks_processing.go:69] worker 13 listening for tasks. I0505 13:05:41.752112 1 tasks_processing.go:69] worker 18 listening for tasks. I0505 13:05:41.752112 1 tasks_processing.go:69] worker 59 listening for tasks. I0505 13:05:41.752118 1 tasks_processing.go:69] worker 54 listening for tasks. I0505 13:05:41.751997 1 tasks_processing.go:71] worker 21 working on machine_sets task. I0505 13:05:41.752069 1 tasks_processing.go:69] worker 38 listening for tasks. I0505 13:05:41.752065 1 tasks_processing.go:69] worker 7 listening for tasks. I0505 13:05:41.752121 1 tasks_processing.go:71] worker 9 working on install_plans task. I0505 13:05:41.752206 1 tasks_processing.go:71] worker 50 working on oauths task. I0505 13:05:41.752211 1 tasks_processing.go:71] worker 33 working on image_pruners task. I0505 13:05:41.752222 1 tasks_processing.go:71] worker 7 working on sap_config task. I0505 13:05:41.752225 1 tasks_processing.go:71] worker 23 working on openstack_dataplanenodesets task. I0505 13:05:41.752236 1 tasks_processing.go:71] worker 11 working on support_secret task. I0505 13:05:41.752120 1 tasks_processing.go:69] worker 56 listening for tasks. I0505 13:05:41.752125 1 tasks_processing.go:69] worker 57 listening for tasks. I0505 13:05:41.752135 1 tasks_processing.go:71] worker 3 working on nodes task. I0505 13:05:41.752258 1 tasks_processing.go:71] worker 25 working on validating_webhook_configurations task. I0505 13:05:41.752257 1 tasks_processing.go:71] worker 56 working on certificate_signing_requests task. I0505 13:05:41.752276 1 tasks_processing.go:71] worker 22 working on storage_classes task. I0505 13:05:41.752287 1 tasks_processing.go:71] worker 42 working on cost_management_metrics_configs task. I0505 13:05:41.752137 1 tasks_processing.go:69] worker 62 listening for tasks. I0505 13:05:41.751986 1 tasks_processing.go:69] worker 20 listening for tasks. I0505 13:05:41.752296 1 tasks_processing.go:71] worker 40 working on infrastructures task. I0505 13:05:41.752129 1 tasks_processing.go:69] worker 53 listening for tasks. I0505 13:05:41.752292 1 tasks_processing.go:71] worker 34 working on operators task. I0505 13:05:41.752305 1 tasks_processing.go:71] worker 51 working on service_accounts task. I0505 13:05:41.752310 1 tasks_processing.go:71] worker 37 working on networks task. I0505 13:05:41.752305 1 tasks_processing.go:71] worker 13 working on active_alerts task. I0505 13:05:41.752295 1 tasks_processing.go:71] worker 19 working on nodenetworkconfigurationpolicies task. I0505 13:05:41.752437 1 tasks_processing.go:71] worker 61 working on storage_cluster task. I0505 13:05:41.752488 1 tasks_processing.go:71] worker 44 working on openstack_version task. I0505 13:05:41.752535 1 tasks_processing.go:71] worker 15 working on pod_network_connectivity_checks task. I0505 13:05:41.752578 1 tasks_processing.go:71] worker 8 working on overlapping_namespace_uids task. I0505 13:05:41.752447 1 tasks_processing.go:71] worker 18 working on sap_datahubs task. I0505 13:05:41.752657 1 tasks_processing.go:71] worker 49 working on monitoring_persistent_volumes task. I0505 13:05:41.752491 1 tasks_processing.go:71] worker 47 working on operators_pods_and_events task. I0505 13:05:41.752754 1 tasks_processing.go:71] worker 36 working on ceph_cluster task. I0505 13:05:41.752818 1 tasks_processing.go:71] worker 45 working on qemu_kubevirt_launcher_logs task. I0505 13:05:41.752911 1 tasks_processing.go:71] worker 35 working on openshift_machine_api_events task. I0505 13:05:41.752937 1 tasks_processing.go:71] worker 46 working on proxies task. I0505 13:05:41.752971 1 tasks_processing.go:71] worker 28 working on feature_gates task. I0505 13:05:41.752452 1 tasks_processing.go:71] worker 59 working on aggregated_monitoring_cr_names task. I0505 13:05:41.752302 1 tasks_processing.go:71] worker 16 working on machine_healthchecks task. I0505 13:05:41.752455 1 tasks_processing.go:71] worker 54 working on dvo_metrics task. I0505 13:05:41.752459 1 tasks_processing.go:71] worker 38 working on openstack_controlplanes task. I0505 13:05:41.752463 1 tasks_processing.go:71] worker 31 working on silenced_alerts task. I0505 13:05:41.752464 1 tasks_processing.go:71] worker 24 working on cluster_apiserver task. I0505 13:05:41.754139 1 tasks_processing.go:71] worker 57 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0505 13:05:41.754277 1 tasks_processing.go:71] worker 62 working on mutating_webhook_configurations task. I0505 13:05:41.754318 1 tasks_processing.go:71] worker 53 working on schedulers task. I0505 13:05:41.754330 1 tasks_processing.go:71] worker 20 working on ingress_certificates task. I0505 13:05:41.752472 1 tasks_processing.go:71] worker 52 working on version task. I0505 13:05:41.752470 1 tasks_processing.go:71] worker 27 working on ingress task. I0505 13:05:41.752469 1 tasks_processing.go:71] worker 17 working on tsdb_status task. I0505 13:05:41.752475 1 tasks_processing.go:71] worker 55 working on crds task. I0505 13:05:41.752479 1 tasks_processing.go:71] worker 58 working on clusterroles task. I0505 13:05:41.752485 1 tasks_processing.go:71] worker 43 working on config_maps task. I0505 13:05:41.752486 1 tasks_processing.go:71] worker 30 working on image task. I0505 13:05:41.752502 1 tasks_processing.go:71] worker 39 working on machine_configs task. I0505 13:05:41.752507 1 tasks_processing.go:71] worker 26 working on machine_config_pools task. I0505 13:05:41.752516 1 tasks_processing.go:71] worker 60 working on openstack_dataplanedeployments task. I0505 13:05:41.752297 1 tasks_processing.go:71] worker 41 working on openshift_logging task. I0505 13:05:41.752521 1 tasks_processing.go:71] worker 63 working on container_runtime_configs task. I0505 13:05:41.752525 1 tasks_processing.go:71] worker 48 working on jaegers task. I0505 13:05:41.752530 1 tasks_processing.go:71] worker 29 working on authentication task. I0505 13:05:41.752531 1 tasks_processing.go:71] worker 12 working on lokistack task. I0505 13:05:41.752299 1 tasks_processing.go:71] worker 14 working on pdbs task. I0505 13:05:41.752480 1 tasks_processing.go:71] worker 6 working on node_logs task. I0505 13:05:41.757723 1 tasks_processing.go:71] worker 5 working on olm_operators task. I0505 13:05:41.757731 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 5.741731ms to process 0 records I0505 13:05:41.757830 1 tasks_processing.go:74] worker 4 stopped. E0505 13:05:41.757850 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0505 13:05:41.757863 1 gather.go:177] gatherer "clusterconfig" function "machines" took 5.850947ms to process 0 records E0505 13:05:41.758934 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.141.67:9091: connect: connection refused E0505 13:05:41.758938 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.141.67:9091: connect: connection refused I0505 13:05:41.758954 1 tasks_processing.go:74] worker 17 stopped. E0505 13:05:41.758961 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.141.67:9091: connect: connection refused I0505 13:05:41.758970 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 4.261033ms to process 0 records E0505 13:05:41.758978 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.141.67:9091: connect: connection refused I0505 13:05:41.758992 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.809723ms to process 0 records I0505 13:05:41.759004 1 tasks_processing.go:74] worker 2 stopped. I0505 13:05:41.759108 1 tasks_processing.go:74] worker 19 stopped. I0505 13:05:41.759126 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 6.740299ms to process 0 records I0505 13:05:41.759327 1 tasks_processing.go:74] worker 61 stopped. I0505 13:05:41.759334 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 6.835302ms to process 0 records I0505 13:05:41.759340 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 7.15322ms to process 0 records I0505 13:05:41.759345 1 tasks_processing.go:74] worker 21 stopped. I0505 13:05:41.762742 1 tasks_processing.go:74] worker 44 stopped. I0505 13:05:41.762751 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 10.201466ms to process 0 records I0505 13:05:41.762765 1 tasks_processing.go:74] worker 23 stopped. I0505 13:05:41.762778 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 10.526248ms to process 0 records I0505 13:05:41.762792 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 10.494506ms to process 0 records E0505 13:05:41.762799 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0505 13:05:41.762805 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 10.23701ms to process 0 records I0505 13:05:41.762811 1 tasks_processing.go:74] worker 15 stopped. I0505 13:05:41.762815 1 tasks_processing.go:74] worker 42 stopped. I0505 13:05:41.764396 1 tasks_processing.go:74] worker 36 stopped. I0505 13:05:41.764408 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 11.621422ms to process 0 records I0505 13:05:41.777013 1 tasks_processing.go:74] worker 38 stopped. I0505 13:05:41.777025 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 23.38687ms to process 0 records I0505 13:05:41.777032 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 24.89176ms to process 0 records I0505 13:05:41.777037 1 tasks_processing.go:74] worker 1 stopped. I0505 13:05:41.777065 1 tasks_processing.go:74] worker 0 stopped. I0505 13:05:41.777073 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 24.921155ms to process 0 records E0505 13:05:41.777094 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0505 13:05:41.777099 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 23.871461ms to process 0 records I0505 13:05:41.777106 1 tasks_processing.go:74] worker 16 stopped. I0505 13:05:41.777330 1 tasks_processing.go:74] worker 37 stopped. I0505 13:05:41.777403 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0505 13:05:41.777423 1 controller.go:326] The initial operator extension status is healthy I0505 13:05:41.777435 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0505 13:05:41.777443 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0505 13:05:41.777448 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0505 13:05:41.777471 1 controller.go:489] The operator is still being initialized I0505 13:05:41.777483 1 controller.go:512] The operator is healthy I0505 13:05:41.777518 1 recorder.go:75] Recording config/network with fingerprint=aeefac480e4c5c738c68d1f191097d40b3a2e41ccebb7cc6e2cce308300543ff I0505 13:05:41.777536 1 gather.go:177] gatherer "clusterconfig" function "networks" took 25.005809ms to process 1 records I0505 13:05:41.777556 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0505 13:05:41.777558 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0505 13:05:41.777713 1 operator.go:288] started I0505 13:05:41.777741 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0505 13:05:41.780011 1 tasks_processing.go:74] worker 46 stopped. I0505 13:05:41.780112 1 recorder.go:75] Recording config/proxy with fingerprint=d7a895d28d76a0948a332e42442f0fee201ce1e79fff744e5fbadc13eaedbaed I0505 13:05:41.780129 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 27.051269ms to process 1 records I0505 13:05:41.784268 1 tasks_processing.go:74] worker 7 stopped. I0505 13:05:41.784285 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 32.036206ms to process 0 records I0505 13:05:41.784295 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 31.624417ms to process 0 records I0505 13:05:41.784300 1 tasks_processing.go:74] worker 18 stopped. I0505 13:05:41.784492 1 tasks_processing.go:74] worker 50 stopped. I0505 13:05:41.784724 1 recorder.go:75] Recording config/oauth with fingerprint=df553c74ab5f8711dae1b5bfb990f57a68e4dcdb879b74430ed12b0ae4216dc9 I0505 13:05:41.784738 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 32.272912ms to process 1 records I0505 13:05:41.784824 1 tasks_processing.go:74] worker 33 stopped. I0505 13:05:41.784950 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=71ac98a73f809f1af54cea1ac444e98eb735f3e1acf030016ae68f43d549cef1 I0505 13:05:41.784960 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 32.354615ms to process 1 records I0505 13:05:41.785180 1 tasks_processing.go:74] worker 40 stopped. I0505 13:05:41.785661 1 recorder.go:75] Recording config/infrastructure with fingerprint=f6aee3e71e6c70db5e5fc5e1e9913c1e0475b8c5a74fe464f1e594b78a4c17f9 I0505 13:05:41.785679 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 32.872464ms to process 1 records I0505 13:05:41.789263 1 tasks_processing.go:74] worker 41 stopped. I0505 13:05:41.789277 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 33.241444ms to process 0 records I0505 13:05:41.789294 1 tasks_processing.go:74] worker 11 stopped. I0505 13:05:41.789306 1 gather_logs.go:145] no pods in namespace were found E0505 13:05:41.789306 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0505 13:05:41.789320 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 37.047879ms to process 0 records I0505 13:05:41.789327 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 36.484913ms to process 0 records I0505 13:05:41.789334 1 tasks_processing.go:74] worker 45 stopped. I0505 13:05:41.789401 1 tasks_processing.go:74] worker 49 stopped. I0505 13:05:41.789411 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 36.734336ms to process 0 records I0505 13:05:41.789428 1 tasks_processing.go:74] worker 22 stopped. I0505 13:05:41.789532 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=1834be67e12b4c154d7b8510c065ec719a380ffbe4e554a42b34ff0d143e4cb7 I0505 13:05:41.789557 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=f2c4f013e2ac0973fcb231091ed36925a5e8fba55339f643d36c95c8eeb7f1e7 I0505 13:05:41.789565 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 37.139013ms to process 2 records I0505 13:05:41.791609 1 tasks_processing.go:74] worker 12 stopped. I0505 13:05:41.791625 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 35.14599ms to process 0 records I0505 13:05:41.791900 1 tasks_processing.go:74] worker 48 stopped. I0505 13:05:41.791910 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 35.58272ms to process 0 records I0505 13:05:41.791924 1 tasks_processing.go:74] worker 35 stopped. I0505 13:05:41.791938 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 38.995272ms to process 0 records I0505 13:05:41.792141 1 tasks_processing.go:74] worker 24 stopped. I0505 13:05:41.792273 1 recorder.go:75] Recording config/apiserver with fingerprint=dd652dd928e4f2aa3a1618e24a4f32867e7d75f66b1c403d188048b4f4346a2a I0505 13:05:41.792290 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 38.108939ms to process 1 records I0505 13:05:41.792375 1 tasks_processing.go:74] worker 60 stopped. I0505 13:05:41.792389 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 36.456349ms to process 0 records I0505 13:05:41.792469 1 tasks_processing.go:74] worker 63 stopped. I0505 13:05:41.792484 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 36.346738ms to process 0 records I0505 13:05:41.792692 1 tasks_processing.go:74] worker 30 stopped. I0505 13:05:41.793182 1 recorder.go:75] Recording config/image with fingerprint=57e9dfb36a41c00702aaf4b37db49a865b88764d6ee6105d26eb10f550dab2b4 I0505 13:05:41.793214 1 gather.go:177] gatherer "clusterconfig" function "image" took 36.912682ms to process 1 records I0505 13:05:41.793718 1 tasks_processing.go:74] worker 27 stopped. I0505 13:05:41.794147 1 recorder.go:75] Recording config/ingress with fingerprint=b17c23ca79f681791d6d27c7aa77c6fbcf86fe4eba8a9035defc38c2953adbfa I0505 13:05:41.794211 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 39.139424ms to process 1 records I0505 13:05:41.794398 1 tasks_processing.go:74] worker 29 stopped. I0505 13:05:41.794828 1 recorder.go:75] Recording config/authentication with fingerprint=d2310e1ca6fabe0235d65a18e46b054eb3670e22fe0dc56fda91bd52ce090a0d I0505 13:05:41.794853 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 37.944765ms to process 1 records I0505 13:05:41.794872 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 38.291792ms to process 0 records I0505 13:05:41.794884 1 tasks_processing.go:74] worker 6 stopped. I0505 13:05:41.795235 1 tasks_processing.go:74] worker 26 stopped. I0505 13:05:41.795286 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 39.386597ms to process 0 records I0505 13:05:41.795364 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0505 13:05:41.795386 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 42.655009ms to process 1 records I0505 13:05:41.795444 1 tasks_processing.go:74] worker 8 stopped. I0505 13:05:41.795497 1 tasks_processing.go:74] worker 62 stopped. I0505 13:05:41.796020 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=7b142bc1b14b9ff945ae630d09e5d86f6a7bec029a9c80ed98153e21e9349171 I0505 13:05:41.796066 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=35b67948d5e26df07306cbdb274464325bf86b87df8bd28935864f70dcf0abe8 I0505 13:05:41.796130 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=692ae0ee1358eecef6645fd32f0e397570f97d7b808e53b6b4c979bf6e179771 I0505 13:05:41.796190 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 41.07135ms to process 3 records I0505 13:05:41.796234 1 tasks_processing.go:74] worker 53 stopped. I0505 13:05:41.797517 1 controller.go:212] Source scaController *sca.Controller is not ready I0505 13:05:41.797576 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0505 13:05:41.797589 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0505 13:05:41.797593 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0505 13:05:41.797597 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0505 13:05:41.797622 1 controller.go:489] The operator is still being initialized I0505 13:05:41.797666 1 controller.go:512] The operator is healthy I0505 13:05:41.797816 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=8fd73c9a4e3e429c47b838e668686713be69313cc21e411610e970f558feea77 I0505 13:05:41.797837 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 41.243731ms to process 1 records I0505 13:05:41.797934 1 tasks_processing.go:74] worker 28 stopped. I0505 13:05:41.797964 1 recorder.go:75] Recording config/featuregate with fingerprint=af119e5592c2d20a6fad5ce0bb49f4abc16935fa70bbe23111a3ce92b589721e I0505 13:05:41.797975 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 44.115399ms to process 1 records I0505 13:05:41.798217 1 tasks_processing.go:74] worker 32 stopped. I0505 13:05:41.798707 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=a724fe96cbcefb3a70b09c34db96afc1ec90f3cf8eb84d0971b7dde2a65cce1d I0505 13:05:41.798724 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 46.21527ms to process 1 records I0505 13:05:41.798820 1 tasks_processing.go:74] worker 3 stopped. I0505 13:05:41.799285 1 recorder.go:75] Recording config/node/ip-10-0-0-107.ec2.internal with fingerprint=eccb1b4d3214dfb95ce70c327265349920d0889f0348e0a3cee2955f8dd83262 I0505 13:05:41.799374 1 recorder.go:75] Recording config/node/ip-10-0-1-131.ec2.internal with fingerprint=d5afa286df90e6be8e5fa2da4c58895903e794443a9b83ff3e7fbde7416a92d3 I0505 13:05:41.799455 1 recorder.go:75] Recording config/node/ip-10-0-2-123.ec2.internal with fingerprint=b2bddf28ff3597edfe38f87e870b2a3f20a1d73761456ea89c96c91ae982c0de I0505 13:05:41.799468 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 46.554116ms to process 3 records I0505 13:05:41.799512 1 tasks_processing.go:74] worker 25 stopped. I0505 13:05:41.799577 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=2171b7c6b134f4a776d7df3bf9370d9495baa64cd42ada9f471f5ee92b99dbf2 W0505 13:05:41.799613 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 13:05:41.799653 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=f420a03fe6c5cc15f9c21805f539c15522db5526d2ac8bf58a27e4e37cc34938 I0505 13:05:41.799802 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=d0fba24dc10e7fa833a0bfc649daa51c4782e70e4de6a4dfa78a19857129d776 I0505 13:05:41.799865 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=80d4692d41ac28fdeedc7b2ec424da672971594ff3f84388c76d81503d6fcb0a I0505 13:05:41.799911 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=02d55495fec2cc31decacd9203ae0c1c85ab71ac4955ad8822ca2531022212ea I0505 13:05:41.799957 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=0e699fe5add4441440e494db5df2a6595fd5ed5239a26c2da37bcdcea45c8a5e I0505 13:05:41.799987 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=1397f66db555359de07e3e471eff40ae8979472a21f7fbaf6162becab0511451 I0505 13:05:41.800016 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=40a2d974e48dae56ae50e2bdaf9662c78ecb4b52f1c4553a3b8100f06d3d8eb3 I0505 13:05:41.800048 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=24435d4531dfee4659c0cc49916a8e307ad4e9cca2f3db58aa0383de053855b9 I0505 13:05:41.800102 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=7a5ee7cc2425ec9e040ecae4b8c0c8a551c78eb35154ae265b403791ed59f86c I0505 13:05:41.800130 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=227f02e7eba8b110a21ac411c188155faf9182bd3acef5089370e7893e07a57a I0505 13:05:41.800153 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=298246c1f9f4bd815e94382ee7e9572153370fe32e08d321f19189def16391f3 I0505 13:05:41.800177 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=35972783adb33db28d8018fd3656cb45cf1ffaa16feee71c77da37db70542852 I0505 13:05:41.800185 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 46.656984ms to process 13 records I0505 13:05:41.800271 1 tasks_processing.go:74] worker 14 stopped. I0505 13:05:41.800273 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=c061c11e3838bbc33e6e105515ec4dc1cc171ed26b185b8e8afbafb8483080e1 I0505 13:05:41.800303 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=33ee98ef26cd47b2f8d7e2c03d2cbbfabee35b781e0cb04a8086165cd054f24f I0505 13:05:41.800323 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=1215412c44223178ee1ab670d17c3c2f0ee534a43a0f5ac8954c643c82612231 I0505 13:05:41.800339 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=7120ee655c5ec279932007ee01d0e0e1419552f2609f6b2c92b55a765af0b13c I0505 13:05:41.800354 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=810306ea645138f370b3a8a91a7f1eb25db5cec889e3c89a364d309c16f77c8e I0505 13:05:41.800368 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=0f0ac72a09639dccb18762d62d3dc44439dd04b17f5329bd60684e533b1baa4c I0505 13:05:41.800382 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=1994c541a4721c0b7e5ebcaa2a73d68ed43a8308dd5d3d84442d8585ff4df63a I0505 13:05:41.800397 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=777d73a89e2bda0f8a1259fae33d9c63dc3162b8ffba85c65d82d6819f36691c I0505 13:05:41.800410 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=e9073f239fdd88fe2dd6cd011c8925d23fe0eb5a64405238686d71206c05b6c7 I0505 13:05:41.800425 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=b3279e794f4b292133f785667f4da8a3471040df5b8ea8d9215dd58fb6f78296 I0505 13:05:41.800438 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=ae3546b49a569514db09e2606fa9dcddf5300b71ed17007bf81787af4401c121 I0505 13:05:41.800444 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 42.809637ms to process 11 records I0505 13:05:41.800899 1 tasks_processing.go:74] worker 56 stopped. I0505 13:05:41.800916 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 48.626593ms to process 0 records I0505 13:05:41.807415 1 tasks_processing.go:74] worker 59 stopped. I0505 13:05:41.807433 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 54.288643ms to process 0 records I0505 13:05:41.807799 1 tasks_processing.go:74] worker 58 stopped. I0505 13:05:41.807982 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=d817ed6bd07203f7db0095fa991da7303a60d6acd062dde8ff469242f6dfe0f7 I0505 13:05:41.808066 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=8223606204da7506daf3707b87f16d6ee2cfd9e8ed1ca04854fa7537388de293 I0505 13:05:41.808088 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 52.549221ms to process 2 records I0505 13:05:41.809592 1 tasks_processing.go:74] worker 52 stopped. I0505 13:05:41.809868 1 recorder.go:75] Recording config/version with fingerprint=366dd55022aabb8506281d57928a9fe7d2d3a6043261035b9f7b8072353bb8f3 I0505 13:05:41.809882 1 recorder.go:75] Recording config/id with fingerprint=b9dad54cb66a93ffd5ea8c79bd5b1cf624907b3291c732b19355d8cfc7699cd0 I0505 13:05:41.809890 1 gather.go:177] gatherer "clusterconfig" function "version" took 55.198949ms to process 2 records I0505 13:05:41.809975 1 tasks_processing.go:74] worker 55 stopped. I0505 13:05:41.810377 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=87365ca4e0476ed3d0b7cd4923336213c74a2678d7e16948f8dfad80878c38f6 I0505 13:05:41.810595 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=61381132a5effca487563aec64778781da1e8545c2efcc8dfa9917cf6b040cd2 I0505 13:05:41.810608 1 gather.go:177] gatherer "clusterconfig" function "crds" took 54.695884ms to process 2 records I0505 13:05:41.811301 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0505 13:05:41.811379 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0505 13:05:41.811403 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0505 13:05:41.814656 1 tasks_processing.go:74] worker 5 stopped. I0505 13:05:41.814709 1 recorder.go:75] Recording config/olm_operators with fingerprint=5db7374bf80262f7f7aae74f3ad95f08f8b840d3fd485c45dffe7419c8ee83b4 I0505 13:05:41.814720 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 56.918497ms to process 1 records I0505 13:05:41.815929 1 tasks_processing.go:74] worker 39 stopped. I0505 13:05:41.815961 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0505 13:05:41.815977 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 60.12388ms to process 1 records I0505 13:05:41.816313 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0505 13:05:41.819185 1 base_controller.go:82] Caches are synced for ConfigController I0505 13:05:41.819198 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0505 13:05:41.822159 1 tasks_processing.go:74] worker 57 stopped. I0505 13:05:41.822199 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 67.950088ms to process 0 records I0505 13:05:41.825784 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 13:05:41.826156 1 tasks_processing.go:74] worker 10 stopped. I0505 13:05:41.827387 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-w5p29 with fingerprint=3b1191a730d61fc101acfaea145f89f4bb7ab8edaf77f59886de4cdccacbcc2e I0505 13:05:41.827474 1 recorder.go:75] Recording config/running_containers with fingerprint=5091efe43f8b1dcbb61fde99a48fa894d246fa06597e390d8697a4a4b680572e I0505 13:05:41.827485 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 74.139527ms to process 2 records I0505 13:05:41.831932 1 tasks_processing.go:74] worker 20 stopped. E0505 13:05:41.831947 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0505 13:05:41.831955 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q3pe17u55dtlb0jnobhvmj01dk6ubgf-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q3pe17u55dtlb0jnobhvmj01dk6ubgf-primary-cert-bundle-secret" not found I0505 13:05:41.832002 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=858f45e50fa2966eca15561332d970f6fa1fff673e81f23efcaaeb8f37c9f2bb I0505 13:05:41.832015 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 77.581841ms to process 1 records I0505 13:05:41.835849 1 tasks_processing.go:74] worker 43 stopped. E0505 13:05:41.835861 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0505 13:05:41.835868 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0505 13:05:41.835872 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0505 13:05:41.835881 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0505 13:05:41.835921 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0505 13:05:41.835929 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0505 13:05:41.835933 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0505 13:05:41.835942 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0505 13:05:41.835980 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0505 13:05:41.835986 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0505 13:05:41.835991 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 80.110345ms to process 7 records I0505 13:05:41.861507 1 cluster_transfer.go:102] no available accepted cluster transfer I0505 13:05:41.861523 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0505 13:05:41.878797 1 base_controller.go:82] Caches are synced for LoggingSyncer I0505 13:05:41.878810 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0505 13:05:41.879811 1 gather_cluster_operator_pods_and_events.go:121] Found 12 pods with 15 containers I0505 13:05:41.879829 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1677721 bytes I0505 13:05:41.881797 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-27rkk pod in namespace openshift-dns (previous: false). I0505 13:05:41.888317 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0505 13:05:41.894066 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0505 13:05:42.018801 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-27rkk pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-27rkk\" is waiting to start: ContainerCreating" I0505 13:05:42.018819 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-27rkk\" is waiting to start: ContainerCreating" I0505 13:05:42.018827 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-27rkk pod in namespace openshift-dns (previous: false). I0505 13:05:42.044425 1 sca.go:266] etc-pki-entitlement secret successfully created I0505 13:05:42.044445 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated I0505 13:05:42.212973 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-27rkk pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-27rkk\" is waiting to start: ContainerCreating" I0505 13:05:42.212989 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-27rkk\" is waiting to start: ContainerCreating" I0505 13:05:42.213018 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-w922b pod in namespace openshift-dns (previous: false). I0505 13:05:42.412297 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-w922b pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-w922b\" is waiting to start: ContainerCreating" I0505 13:05:42.412313 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-w922b\" is waiting to start: ContainerCreating" I0505 13:05:42.412321 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-w922b pod in namespace openshift-dns (previous: false). I0505 13:05:42.612016 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-w922b pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-w922b\" is waiting to start: ContainerCreating" I0505 13:05:42.612033 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-w922b\" is waiting to start: ContainerCreating" I0505 13:05:42.612045 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-z5k5d pod in namespace openshift-dns (previous: false). I0505 13:05:42.788633 1 tasks_processing.go:74] worker 31 stopped. I0505 13:05:42.788653 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0505 13:05:42.788664 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 1.034861226s to process 1 records I0505 13:05:42.788705 1 tasks_processing.go:74] worker 13 stopped. I0505 13:05:42.788753 1 recorder.go:75] Recording config/alerts with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 W0505 13:05:42.788770 1 gather.go:155] issue recording gatherer "clusterconfig" function "active_alerts" result "config/alerts.json" because of the warning: warning: the record with the same fingerprint "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945" was already recorded at path "config/namespaces_with_overlapping_uids.json", recording another one with a different path "config/alerts.json" I0505 13:05:42.788783 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 1.036348018s to process 1 records W0505 13:05:42.797737 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 13:05:42.813885 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-z5k5d pod in namespace openshift-dns (previous: false). I0505 13:05:43.013515 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-6k7th pod in namespace openshift-dns (previous: false). I0505 13:05:43.214105 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-tkv4j pod in namespace openshift-dns (previous: false). I0505 13:05:43.411856 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-vbr4s pod in namespace openshift-dns (previous: false). I0505 13:05:43.614313 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0505 13:05:43.614332 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5498d7b5c9-lr8ls pod in namespace openshift-ingress (previous: false). W0505 13:05:43.797203 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 13:05:43.815311 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5498d7b5c9-qfxrq pod in namespace openshift-ingress (previous: false). I0505 13:05:44.015954 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-df4986679-bgltq pod in namespace openshift-ingress (previous: false). I0505 13:05:44.215227 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-5kd7h pod in namespace openshift-ingress-canary (previous: false). I0505 13:05:44.226349 1 tasks_processing.go:74] worker 34 stopped. I0505 13:05:44.226476 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=f549fcbcdaf5c3e37ea3dcfa05a325567344e2c8df4d93da71122fbe98e79159 I0505 13:05:44.226528 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0505 13:05:44.226550 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=cdb4ee9966301c01bc824e4378b569490ad3367724d6e63eaaec12381033052a I0505 13:05:44.226561 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0505 13:05:44.226586 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=417e8b64a3de9d524c481d4899bc1f2fdd0b02732a0e7a8d1252e2c7ebf7676c I0505 13:05:44.226602 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0505 13:05:44.226623 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=a43e84105894067fc11ed1f4ee6b1eaecb1ff7d49f96cb0561483f98154c8d1d I0505 13:05:44.226653 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=95aff58aebe316737dc8da12dd18d1cc0361ffb08b7e0ba89e4d8ae79f229c45 I0505 13:05:44.226677 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=fe479d697b141583fa1c4e1cbc6dd16844209b61b761daedc1ccfb50807fc81f I0505 13:05:44.226707 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=2358e43ab33486665a17557e6e9682a007a5735970826b8e14b5d0acee2f61a8 I0505 13:05:44.226967 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=784f3dcec8bf9d81da5e3ef8663951070227ca83400f4d6d6e2615422a68c047 I0505 13:05:44.226988 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=c1a5c5e0834c7dd8c206cca8a0a1bc5afed985c61f42f232d1ca2389835c2f2d I0505 13:05:44.226997 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0505 13:05:44.227013 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=1141d8d05db335dcfa235301a7caa20bced16fa60bbf85f23f3b61ef2e4f5d44 I0505 13:05:44.227024 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0505 13:05:44.227040 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=7e4aaa5534bb79dc9dc46e8a8ac4aeea445837ea594974a153a24ebe8796262c I0505 13:05:44.227049 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0505 13:05:44.227068 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=b688171567353d4e0b1bb7cea212ddd246ae43629a01b03142a27085b1116089 I0505 13:05:44.227103 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0505 13:05:44.227127 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=855680411a66e2fe669c102d45256116e465f58069d356263824575f22baa760 I0505 13:05:44.227228 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=12147458046f7d26e6a4681edb57b058d3b14ad1c9fb3eff936cd7726e884b53 I0505 13:05:44.227238 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0505 13:05:44.227246 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0505 13:05:44.227282 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0505 13:05:44.227304 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=9bd2ad91555da8d6a46889fe80a179611584877a63ac658c9acf3ee4c7c8586a I0505 13:05:44.227328 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=150874fd1a41f3b56ec9ab7883766e202c3e116628cc1b7b0b473dab61cca326 I0505 13:05:44.227338 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0505 13:05:44.227354 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=c522f72a004e3d667f695f6a70dec0b4d43ad4f3c8bcbc81a09470ddfa0d0e76 I0505 13:05:44.227364 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0505 13:05:44.227382 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=e8eae5451b22cc85a0dad21b03992b536a482dfdd7cf9abeb4036fb1f2f678b1 I0505 13:05:44.227392 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0505 13:05:44.227404 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=0aef41a8422ae5d50b91c3cc0493f3724d9b39db3aea94987c4439eacd886208 I0505 13:05:44.227421 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=3b87f3bbcb1acf541e0c67072eb069c1c292efaa68b3cc5c1586822a96003c82 I0505 13:05:44.227437 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=c47a1bc34c3766da5898aba177781cd9b1ce55456fe8ca1bbe1457ad191dc1d6 I0505 13:05:44.227458 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=48cab8979b9c1d6755e14c2f9d009c61884452d4a862c6472ba87ff61f9d9adc I0505 13:05:44.227468 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0505 13:05:44.227491 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=f66c44f9e6e0bb256579e05c8fbe9750fe9d0f8b610aabb887d7421c3d368ca7 I0505 13:05:44.227508 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0505 13:05:44.227517 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0505 13:05:44.227525 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.474033756s to process 39 records I0505 13:05:44.412751 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-5kd7h pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-5kd7h\" is waiting to start: ContainerCreating" I0505 13:05:44.412770 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-5kd7h\" is waiting to start: ContainerCreating" I0505 13:05:44.412781 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-f7vdl pod in namespace openshift-ingress-canary (previous: false). I0505 13:05:44.616128 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-mkg8m pod in namespace openshift-ingress-canary (previous: false). W0505 13:05:44.797298 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0505 13:05:44.811810 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-mkg8m pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mkg8m\" is waiting to start: ContainerCreating" I0505 13:05:44.811835 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mkg8m\" is waiting to start: ContainerCreating" I0505 13:05:44.811853 1 tasks_processing.go:74] worker 47 stopped. I0505 13:05:44.811939 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=11bca195df96058db6d536531b723b44d98b755bf68140e8e2527d57889fea7d I0505 13:05:44.812008 1 recorder.go:75] Recording events/openshift-dns with fingerprint=a17c00f3992bd8d1b3d568ccfba11b0cf5997cfba616dcca130436c7240dca65 I0505 13:05:44.812044 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=9e299b9410fc594c4cc50e41d213f674de0a5a6ac3aad91595e8e68e98229ab6 I0505 13:05:44.812191 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=7c18f37eac60afaf82370d42c8744be98607ba3ed80d578b6dc7aa3c42b28434 I0505 13:05:44.812223 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=7db454dee0fc1174acbe05fbd76a902528dd273fcbaf72cf113ed374b30c460a I0505 13:05:44.812375 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-27rkk with fingerprint=00174a3ca0b8ebb557ad94af74c9468ac6e2225a17beaa067af7d0149ab38bf6 I0505 13:05:44.812467 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-w922b with fingerprint=6bd12cffd0860b633ab62e83b499874dfde3298bed8b3305af8dc2716255d4ed I0505 13:05:44.812480 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-z5k5d/dns_current.log with fingerprint=e3da201e2bd369677d2bd1cf3b5576f3469293a1a744ca8fb99eb76c1a190dc4 I0505 13:05:44.812487 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-z5k5d/kube-rbac-proxy_current.log with fingerprint=8a8493fbf7101c708bc708b351609109b2d43455eaf7e56877a0fda9f53b9789 I0505 13:05:44.812507 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-6k7th/dns-node-resolver_current.log with fingerprint=d061cbd9388a677aaf5cc9aba8adc3785de8dff9f00449c9d4b986663909ca7a I0505 13:05:44.812524 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-tkv4j/dns-node-resolver_current.log with fingerprint=7d1f25a4deaa510e709fe973828265fe1a2d2cebbfc5a294fb69310c53adb79a I0505 13:05:44.812576 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5498d7b5c9-lr8ls/router_current.log with fingerprint=8f555537df836f1e43db1091d6a15faae57aa940daf7d28bf25cd67887c5e736 I0505 13:05:44.812619 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5498d7b5c9-qfxrq/router_current.log with fingerprint=c1312deefd0de01ef05fe05e91e839c4de0b20bf0d564d440e8231b613d9d901 I0505 13:05:44.812654 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-df4986679-bgltq/router_current.log with fingerprint=9c70bd2dc84dd991d1c5268d8f3923f4af4f6a070cf24c41beea1e773842de96 I0505 13:05:44.812722 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-5kd7h with fingerprint=48c877ce39ec99bbaeecbe1419f0331d3ab0df39d4dfdb3b82c92a8e2d90fe09 I0505 13:05:44.812729 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-f7vdl/serve-healthcheck-canary_current.log with fingerprint=56bbfb7ff125aff17a93e7d9f1c30eed955f80ec7299df2c54c57ef8a5d22360 I0505 13:05:44.812788 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-mkg8m with fingerprint=f7ab755406a0d0d9f373b1bf147cd519859b88a4f4000bfe63cd72210d92893c I0505 13:05:44.812800 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 3.059172972s to process 17 records W0505 13:05:45.798207 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 13:05:46.797726 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0505 13:05:46.797754 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0505 13:05:46.797769 1 tasks_processing.go:74] worker 54 stopped. E0505 13:05:46.797779 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0505 13:05:46.797790 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0505 13:05:46.797799 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.044256134s to process 1 records I0505 13:05:54.607253 1 tasks_processing.go:74] worker 9 stopped. I0505 13:05:54.607295 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0505 13:05:54.607307 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.85503164s to process 1 records I0505 13:05:55.361132 1 tasks_processing.go:74] worker 51 stopped. I0505 13:05:55.361471 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=f5738032f28aea743b389898fadee779c5cd56b79a04dcc121fb2c80f0e9fc94 I0505 13:05:55.361489 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.608807278s to process 1 records E0505 13:05:55.361550 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.609s with: function \"machines\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0505 13:05:55.362651 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0505 13:05:55.362666 1 periodic.go:209] Running workloads gatherer I0505 13:05:55.362679 1 tasks_processing.go:45] number of workers: 2 I0505 13:05:55.362685 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 13:05:55.362689 1 tasks_processing.go:71] worker 1 working on workload_info task. I0505 13:05:55.362699 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 13:05:55.362719 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0505 13:05:55.370440 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-123.ec2.internal... I0505 13:05:55.370445 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-107.ec2.internal... I0505 13:05:55.370458 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-131.ec2.internal... I0505 13:05:55.398159 1 tasks_processing.go:74] worker 0 stopped. I0505 13:05:55.398179 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 35.416571ms to process 0 records I0505 13:05:59.976149 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0505 13:06:00.003515 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0505 13:06:00.013797 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (12ms) I0505 13:06:00.024267 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0505 13:06:00.037487 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (13ms) I0505 13:06:00.047663 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (10ms) I0505 13:06:00.058747 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (11ms) I0505 13:06:00.069197 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (10ms) I0505 13:06:00.080204 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (11ms) I0505 13:06:00.091607 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (11ms) I0505 13:06:00.105032 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (13ms) I0505 13:06:00.119505 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (14ms) I0505 13:06:00.131323 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (12ms) I0505 13:06:00.213693 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (82ms) I0505 13:06:00.316720 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (103ms) I0505 13:06:00.413924 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (97ms) I0505 13:06:00.516658 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (103ms) I0505 13:06:00.613363 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (97ms) I0505 13:06:00.713603 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0505 13:06:00.814239 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (101ms) I0505 13:06:00.916612 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (102ms) I0505 13:06:01.017011 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0505 13:06:01.113753 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (97ms) I0505 13:06:01.216011 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (102ms) I0505 13:06:01.314487 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (98ms) I0505 13:06:01.413302 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (99ms) I0505 13:06:01.513395 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (100ms) I0505 13:06:01.616419 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (103ms) I0505 13:06:01.714250 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (98ms) I0505 13:06:01.813000 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (99ms) I0505 13:06:01.915563 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (103ms) I0505 13:06:02.014308 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0505 13:06:02.114096 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0505 13:06:02.214045 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0505 13:06:02.319628 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (106ms) I0505 13:06:02.413962 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (94ms) I0505 13:06:02.513847 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (100ms) I0505 13:06:02.613810 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0505 13:06:02.715772 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (102ms) I0505 13:06:02.814181 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (98ms) I0505 13:06:02.913395 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0505 13:06:03.013407 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0505 13:06:03.113601 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0505 13:06:03.215831 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (102ms) I0505 13:06:03.314287 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (98ms) I0505 13:06:03.413220 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0505 13:06:03.514164 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (101ms) I0505 13:06:03.614141 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0505 13:06:03.716662 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (103ms) I0505 13:06:03.814995 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (98ms) I0505 13:06:03.918224 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (103ms) I0505 13:06:04.014335 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (96ms) I0505 13:06:04.014368 1 tasks_processing.go:74] worker 1 stopped. I0505 13:06:04.014862 1 recorder.go:75] Recording config/workload_info with fingerprint=bf8ff563346a0ba0163052b04e75012e5650fb40e1a4296486e76d8ef27eb53c I0505 13:06:04.014889 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.651671753s to process 1 records I0505 13:06:04.015019 1 periodic.go:256] Periodic gather workloads completed in 8.652s I0505 13:06:04.015098 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0505 13:06:04.015135 1 periodic.go:209] Running conditional gatherer I0505 13:06:04.024298 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0505 13:06:04.168606 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0505 13:06:05.253666 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0505 13:06:05.268778 1 conditional_gatherer.go:392] cluster version is '4.20.8' I0505 13:06:05.268810 1 tasks_processing.go:45] number of workers: 3 I0505 13:06:05.268818 1 tasks_processing.go:69] worker 2 listening for tasks. I0505 13:06:05.268822 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0505 13:06:05.268826 1 tasks_processing.go:69] worker 0 listening for tasks. I0505 13:06:05.268836 1 tasks_processing.go:69] worker 1 listening for tasks. I0505 13:06:05.268841 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0505 13:06:05.268844 1 tasks_processing.go:74] worker 1 stopped. I0505 13:06:05.268838 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0505 13:06:05.268916 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0505 13:06:05.268930 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.079µs to process 1 records I0505 13:06:05.268964 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0505 13:06:05.268970 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.002µs to process 1 records I0505 13:06:05.268976 1 tasks_processing.go:74] worker 0 stopped. I0505 13:06:05.269124 1 tasks_processing.go:74] worker 2 stopped. I0505 13:06:05.269136 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 275.618µs to process 0 records I0505 13:06:05.269159 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0505 13:06:05.269177 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0505 13:06:05.301625 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=81b6d237aed13e46270af72a1133ec4d3b5f07639fd4f73d75cb2f980eff67ad I0505 13:06:05.301768 1 diskrecorder.go:70] Writing 128 records to /var/lib/insights-operator/insights-2026-05-05-130605.tar.gz I0505 13:06:05.309874 1 diskrecorder.go:51] Wrote 128 records to disk in 8ms I0505 13:06:05.309905 1 periodic.go:278] Gathering cluster info every 2h0m0s I0505 13:06:05.309919 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0505 13:07:11.752092 1 diskrecorder.go:223] Found files to send: insights-2026-05-05-130605.tar.gz I0505 13:07:11.752122 1 insightsuploader.go:150] Checking archives to upload periodically every 15m31.635536035s I0505 13:07:11.752131 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0505 13:07:11.764471 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0505 13:07:11.978242 1 requests.go:87] Successfully reported id=2026-05-05T13:07:11Z x-rh-insights-request-id=834f9a094628471d9fd99817d1e826b9, wrote=60928 I0505 13:07:11.978308 1 insightsuploader.go:187] Uploaded report successfully in 226.168503ms I0505 13:07:11.978337 1 controller.go:128] Initializing last reported time to 2026-05-05T13:07:11Z I0505 13:07:11.978356 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0505 13:07:11.978365 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0505 13:07:11.978372 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0505 13:07:11.985718 1 controller.go:512] The operator is healthy I0505 13:07:41.785260 1 controller.go:512] The operator is healthy I0505 13:08:13.623554 1 insightsreport.go:137] Pulling report from smart-proxy I0505 13:08:13.623580 1 insightsreport.go:149] Retrieving report I0505 13:08:13.632949 1 requests.go:110] Retrieving report for cluster: 424018d0-c6b7-45bb-8289-e61ce177c8b8 I0505 13:08:13.632962 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/424018d0-c6b7-45bb-8289-e61ce177c8b8/reports I0505 13:08:13.635629 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/424018d0-c6b7-45bb-8289-e61ce177c8b8/reports I0505 13:08:13.838952 1 insightsreport.go:184] Report retrieved I0505 13:08:13.866250 1 insightsreport.go:239] Report retrieved correctly I0505 13:09:41.789272 1 controller.go:512] The operator is healthy I0505 13:10:41.743731 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:10:41.750523 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:10:41.750541 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:10:41.755955 1 secretconfigobserver.go:119] support secret does not exist I0505 13:11:41.785980 1 controller.go:512] The operator is healthy I0505 13:13:41.786246 1 controller.go:512] The operator is healthy I0505 13:15:41.736394 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 13:15:41.742143 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 13:15:41.756292 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:15:41.762361 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:15:41.762380 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:15:41.767646 1 secretconfigobserver.go:119] support secret does not exist I0505 13:15:41.784832 1 controller.go:512] The operator is healthy I0505 13:17:41.802687 1 controller.go:512] The operator is healthy I0505 13:19:41.785157 1 controller.go:512] The operator is healthy I0505 13:20:41.768212 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:20:41.775052 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:20:41.775069 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:20:41.782475 1 secretconfigobserver.go:119] support secret does not exist I0505 13:21:41.789919 1 controller.go:512] The operator is healthy I0505 13:22:48.614868 1 insightsuploader.go:146] Nothing to report since 2026-05-05T13:07:11Z I0505 13:23:41.786273 1 controller.go:512] The operator is healthy I0505 13:25:41.737437 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 13:25:41.782653 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:25:41.785940 1 controller.go:512] The operator is healthy I0505 13:25:41.788689 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:25:41.788708 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:25:41.794264 1 secretconfigobserver.go:119] support secret does not exist I0505 13:27:41.786282 1 controller.go:512] The operator is healthy I0505 13:29:41.786694 1 controller.go:512] The operator is healthy I0505 13:30:41.794534 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:30:41.801472 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:30:41.801488 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:30:41.806895 1 secretconfigobserver.go:119] support secret does not exist I0505 13:31:41.785669 1 controller.go:512] The operator is healthy I0505 13:33:41.788980 1 controller.go:512] The operator is healthy I0505 13:35:41.738377 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 13:35:41.744727 1 configmapobserver.go:84] configmaps "insights-config" not found I0505 13:35:41.786021 1 controller.go:512] The operator is healthy I0505 13:35:41.807102 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:35:41.813407 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:35:41.813426 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:35:41.819433 1 secretconfigobserver.go:119] support secret does not exist I0505 13:37:41.786149 1 controller.go:512] The operator is healthy I0505 13:37:53.940588 1 insightsuploader.go:146] Nothing to report since 2026-05-05T13:07:11Z I0505 13:39:41.792166 1 controller.go:512] The operator is healthy I0505 13:40:41.819905 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0505 13:40:41.827264 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0505 13:40:41.827281 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0505 13:40:41.833619 1 secretconfigobserver.go:119] support secret does not exist I0505 13:41:41.788810 1 controller.go:512] The operator is healthy