I0423 21:01:34.139674 1 cmd.go:253] Using service-serving-cert provided certificates I0423 21:01:34.140132 1 observer_polling.go:159] Starting file observer I0423 21:01:34.321825 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0423 21:01:34.322018 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0423 21:01:34.322675 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0423 21:01:34.322759 1 secure_serving.go:57] Forcing use of http/1.1 only W0423 21:01:34.322779 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0423 21:01:34.322783 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0423 21:01:34.322787 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0423 21:01:34.322791 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0423 21:01:34.322794 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0423 21:01:34.322796 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0423 21:01:34.327003 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0423 21:01:34.327049 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"74754fa1-66c3-4a1c-8d54-6bae8a2b4857", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0423 21:01:34.328911 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0423 21:01:34.328934 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0423 21:01:34.328961 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0423 21:01:34.328960 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0423 21:01:34.328979 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 21:01:34.328985 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 21:01:34.329179 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0423 21:01:34.329566 1 secure_serving.go:213] Serving securely on [::]:8443 I0423 21:01:34.329593 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0423 21:01:34.334598 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0423 21:01:34.334618 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 21:01:34.334738 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0423 21:01:34.340837 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 21:01:34.340854 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 21:01:34.345709 1 secretconfigobserver.go:119] support secret does not exist I0423 21:01:34.351883 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 21:01:34.356337 1 secretconfigobserver.go:119] support secret does not exist I0423 21:01:34.358288 1 recorder.go:161] Pruning old reports every 7h5m36s, max age is 288h0m0s I0423 21:01:34.364177 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0423 21:01:34.364184 1 periodic.go:209] Running clusterconfig gatherer I0423 21:01:34.364193 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0423 21:01:34.364223 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0423 21:01:34.364229 1 insightsreport.go:296] Starting report retriever I0423 21:01:34.364234 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0423 21:01:34.364250 1 tasks_processing.go:45] number of workers: 64 I0423 21:01:34.364277 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 21:01:34.364286 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 21:01:34.364288 1 tasks_processing.go:69] worker 13 listening for tasks. I0423 21:01:34.364291 1 tasks_processing.go:69] worker 25 listening for tasks. I0423 21:01:34.364295 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 21:01:34.364303 1 tasks_processing.go:69] worker 20 listening for tasks. I0423 21:01:34.364298 1 tasks_processing.go:69] worker 3 listening for tasks. I0423 21:01:34.364303 1 tasks_processing.go:69] worker 19 listening for tasks. I0423 21:01:34.364310 1 tasks_processing.go:71] worker 0 working on pod_network_connectivity_checks task. I0423 21:01:34.364314 1 tasks_processing.go:69] worker 5 listening for tasks. I0423 21:01:34.364316 1 tasks_processing.go:69] worker 14 listening for tasks. I0423 21:01:34.364322 1 tasks_processing.go:69] worker 9 listening for tasks. I0423 21:01:34.364326 1 tasks_processing.go:69] worker 10 listening for tasks. I0423 21:01:34.364330 1 tasks_processing.go:69] worker 24 listening for tasks. I0423 21:01:34.364322 1 tasks_processing.go:69] worker 21 listening for tasks. I0423 21:01:34.364335 1 tasks_processing.go:69] worker 22 listening for tasks. I0423 21:01:34.364336 1 tasks_processing.go:71] worker 14 working on infrastructures task. I0423 21:01:34.364340 1 tasks_processing.go:69] worker 23 listening for tasks. I0423 21:01:34.364346 1 tasks_processing.go:69] worker 46 listening for tasks. I0423 21:01:34.364347 1 tasks_processing.go:69] worker 17 listening for tasks. I0423 21:01:34.364352 1 tasks_processing.go:69] worker 11 listening for tasks. I0423 21:01:34.364355 1 tasks_processing.go:69] worker 30 listening for tasks. I0423 21:01:34.364357 1 tasks_processing.go:69] worker 8 listening for tasks. I0423 21:01:34.364362 1 tasks_processing.go:69] worker 15 listening for tasks. I0423 21:01:34.364363 1 tasks_processing.go:69] worker 37 listening for tasks. I0423 21:01:34.364356 1 tasks_processing.go:69] worker 36 listening for tasks. I0423 21:01:34.364368 1 tasks_processing.go:69] worker 26 listening for tasks. I0423 21:01:34.364369 1 tasks_processing.go:69] worker 16 listening for tasks. I0423 21:01:34.364373 1 tasks_processing.go:69] worker 38 listening for tasks. I0423 21:01:34.364315 1 tasks_processing.go:71] worker 3 working on support_secret task. I0423 21:01:34.364377 1 tasks_processing.go:69] worker 27 listening for tasks. I0423 21:01:34.364374 1 tasks_processing.go:69] worker 18 listening for tasks. I0423 21:01:34.364382 1 tasks_processing.go:69] worker 39 listening for tasks. I0423 21:01:34.364344 1 tasks_processing.go:69] worker 6 listening for tasks. I0423 21:01:34.364386 1 tasks_processing.go:69] worker 34 listening for tasks. I0423 21:01:34.364317 1 tasks_processing.go:71] worker 19 working on nodenetworkstates task. I0423 21:01:34.364413 1 tasks_processing.go:69] worker 29 listening for tasks. I0423 21:01:34.364417 1 tasks_processing.go:69] worker 41 listening for tasks. I0423 21:01:34.364418 1 tasks_processing.go:69] worker 49 listening for tasks. I0423 21:01:34.364417 1 tasks_processing.go:69] worker 44 listening for tasks. I0423 21:01:34.364424 1 tasks_processing.go:69] worker 45 listening for tasks. I0423 21:01:34.364427 1 tasks_processing.go:71] worker 1 working on jaegers task. I0423 21:01:34.364423 1 tasks_processing.go:69] worker 40 listening for tasks. I0423 21:01:34.364435 1 tasks_processing.go:69] worker 56 listening for tasks. I0423 21:01:34.364435 1 tasks_processing.go:69] worker 54 listening for tasks. I0423 21:01:34.364449 1 tasks_processing.go:69] worker 60 listening for tasks. I0423 21:01:34.364439 1 tasks_processing.go:69] worker 42 listening for tasks. I0423 21:01:34.364456 1 tasks_processing.go:71] worker 54 working on tsdb_status task. I0423 21:01:34.364462 1 tasks_processing.go:71] worker 42 working on cost_management_metrics_configs task. I0423 21:01:34.364469 1 tasks_processing.go:71] worker 16 working on ingress_certificates task. I0423 21:01:34.364463 1 tasks_processing.go:71] worker 34 working on metrics task. I0423 21:01:34.364305 1 tasks_processing.go:69] worker 4 listening for tasks. I0423 21:01:34.364597 1 tasks_processing.go:71] worker 4 working on authentication task. I0423 21:01:34.364626 1 tasks_processing.go:71] worker 2 working on sap_pods task. I0423 21:01:34.364309 1 tasks_processing.go:71] worker 20 working on container_runtime_configs task. I0423 21:01:34.364426 1 tasks_processing.go:69] worker 53 listening for tasks. I0423 21:01:34.364362 1 tasks_processing.go:69] worker 12 listening for tasks. I0423 21:01:34.364368 1 tasks_processing.go:69] worker 31 listening for tasks. I0423 21:01:34.364374 1 tasks_processing.go:69] worker 32 listening for tasks. I0423 21:01:34.364418 1 tasks_processing.go:69] worker 52 listening for tasks. I0423 21:01:34.364377 1 tasks_processing.go:69] worker 51 listening for tasks. I0423 21:01:34.364380 1 tasks_processing.go:69] worker 33 listening for tasks. I0423 21:01:34.364384 1 tasks_processing.go:69] worker 47 listening for tasks. I0423 21:01:34.364385 1 tasks_processing.go:69] worker 28 listening for tasks. I0423 21:01:34.364682 1 tasks_processing.go:71] worker 56 working on openstack_controlplanes task. I0423 21:01:34.364686 1 tasks_processing.go:71] worker 46 working on oauths task. I0423 21:01:34.364689 1 tasks_processing.go:71] worker 8 working on ceph_cluster task. I0423 21:01:34.364384 1 tasks_processing.go:69] worker 57 listening for tasks. I0423 21:01:34.364411 1 tasks_processing.go:69] worker 43 listening for tasks. I0423 21:01:34.364410 1 tasks_processing.go:69] worker 48 listening for tasks. I0423 21:01:34.364424 1 tasks_processing.go:69] worker 62 listening for tasks. I0423 21:01:34.364428 1 tasks_processing.go:69] worker 58 listening for tasks. I0423 21:01:34.364456 1 tasks_processing.go:71] worker 60 working on service_accounts task. I0423 21:01:34.364721 1 tasks_processing.go:71] worker 38 working on openstack_dataplanenodesets task. I0423 21:01:34.364730 1 tasks_processing.go:71] worker 24 working on ingress task. I0423 21:01:34.364739 1 tasks_processing.go:71] worker 27 working on crds task. I0423 21:01:34.364749 1 tasks_processing.go:71] worker 18 working on machine_healthchecks task. I0423 21:01:34.364749 1 tasks_processing.go:71] worker 25 working on aggregated_monitoring_cr_names task. I0423 21:01:34.364803 1 tasks_processing.go:71] worker 37 working on sap_datahubs task. I0423 21:01:34.364351 1 tasks_processing.go:69] worker 7 listening for tasks. I0423 21:01:34.364675 1 tasks_processing.go:71] worker 40 working on monitoring_persistent_volumes task. I0423 21:01:34.365285 1 tasks_processing.go:69] worker 61 listening for tasks. I0423 21:01:34.365315 1 tasks_processing.go:69] worker 63 listening for tasks. I0423 21:01:34.365326 1 tasks_processing.go:69] worker 59 listening for tasks. I0423 21:01:34.365521 1 tasks_processing.go:71] worker 61 working on certificate_signing_requests task. I0423 21:01:34.365599 1 tasks_processing.go:71] worker 59 working on schedulers task. I0423 21:01:34.365325 1 tasks_processing.go:69] worker 55 listening for tasks. I0423 21:01:34.365831 1 tasks_processing.go:69] worker 50 listening for tasks. I0423 21:01:34.364323 1 tasks_processing.go:71] worker 5 working on config_maps task. I0423 21:01:34.365853 1 tasks_processing.go:71] worker 55 working on openstack_version task. I0423 21:01:34.365794 1 tasks_processing.go:71] worker 26 working on image_pruners task. I0423 21:01:34.365949 1 tasks_processing.go:71] worker 50 working on nodes task. I0423 21:01:34.364807 1 tasks_processing.go:71] worker 36 working on install_plans task. I0423 21:01:34.364799 1 tasks_processing.go:71] worker 15 working on silenced_alerts task. I0423 21:01:34.364725 1 tasks_processing.go:71] worker 9 working on operators task. I0423 21:01:34.366097 1 tasks_processing.go:71] worker 13 working on storage_classes task. I0423 21:01:34.364868 1 tasks_processing.go:71] worker 11 working on dvo_metrics task. I0423 21:01:34.366126 1 tasks_processing.go:71] worker 62 working on olm_operators task. I0423 21:01:34.366148 1 tasks_processing.go:71] worker 63 working on machine_configs task. I0423 21:01:34.364875 1 tasks_processing.go:71] worker 17 working on proxies task. I0423 21:01:34.364884 1 tasks_processing.go:71] worker 22 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0423 21:01:34.364891 1 tasks_processing.go:71] worker 21 working on openshift_logging task. I0423 21:01:34.364907 1 tasks_processing.go:71] worker 7 working on sap_config task. I0423 21:01:34.364900 1 tasks_processing.go:71] worker 10 working on openstack_dataplanedeployments task. I0423 21:01:34.364904 1 tasks_processing.go:71] worker 30 working on machines task. I0423 21:01:34.364912 1 tasks_processing.go:71] worker 53 working on cluster_apiserver task. I0423 21:01:34.364915 1 tasks_processing.go:71] worker 12 working on networks task. I0423 21:01:34.364917 1 tasks_processing.go:71] worker 31 working on image_registries task. I0423 21:01:34.364920 1 tasks_processing.go:71] worker 32 working on openshift_machine_api_events task. I0423 21:01:34.364922 1 tasks_processing.go:71] worker 52 working on qemu_kubevirt_launcher_logs task. I0423 21:01:34.364914 1 tasks_processing.go:71] worker 29 working on version task. I0423 21:01:34.364914 1 tasks_processing.go:71] worker 41 working on container_images task. I0423 21:01:34.364926 1 tasks_processing.go:71] worker 51 working on node_logs task. I0423 21:01:34.364930 1 tasks_processing.go:71] worker 33 working on nodenetworkconfigurationpolicies task. I0423 21:01:34.364932 1 tasks_processing.go:71] worker 47 working on active_alerts task. I0423 21:01:34.364926 1 tasks_processing.go:71] worker 23 working on machine_sets task. I0423 21:01:34.364926 1 tasks_processing.go:71] worker 49 working on lokistack task. I0423 21:01:34.364934 1 tasks_processing.go:71] worker 43 working on machine_autoscalers task. I0423 21:01:34.364935 1 tasks_processing.go:71] worker 48 working on feature_gates task. I0423 21:01:34.364939 1 tasks_processing.go:71] worker 57 working on clusterroles task. I0423 21:01:34.364940 1 tasks_processing.go:71] worker 58 working on overlapping_namespace_uids task. I0423 21:01:34.364939 1 tasks_processing.go:71] worker 28 working on storage_cluster task. I0423 21:01:34.364360 1 tasks_processing.go:69] worker 35 listening for tasks. I0423 21:01:34.364670 1 tasks_processing.go:71] worker 45 working on machine_config_pools task. I0423 21:01:34.364814 1 tasks_processing.go:71] worker 39 working on validating_webhook_configurations task. I0423 21:01:34.364854 1 tasks_processing.go:71] worker 6 working on operators_pods_and_events task. I0423 21:01:34.366410 1 tasks_processing.go:71] worker 44 working on pdbs task. I0423 21:01:34.368384 1 tasks_processing.go:71] worker 35 working on image task. I0423 21:01:34.368845 1 tasks_processing.go:71] worker 0 working on mutating_webhook_configurations task. E0423 21:01:34.369121 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0423 21:01:34.369178 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 4.524565ms to process 0 records I0423 21:01:34.369214 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 4.681919ms to process 0 records I0423 21:01:34.369316 1 tasks_processing.go:74] worker 19 stopped. I0423 21:01:34.369999 1 tasks_processing.go:74] worker 1 stopped. I0423 21:01:34.370018 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 5.557893ms to process 0 records I0423 21:01:34.370760 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0423 21:01:34.370784 1 controller.go:326] The initial operator extension status is healthy I0423 21:01:34.370792 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 21:01:34.370796 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0423 21:01:34.370798 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0423 21:01:34.370813 1 controller.go:489] The operator is still being initialized I0423 21:01:34.370818 1 controller.go:512] The operator is healthy I0423 21:01:34.370953 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0423 21:01:34.371038 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0423 21:01:34.371438 1 operator.go:288] started I0423 21:01:34.371464 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0423 21:01:34.371731 1 tasks_processing.go:74] worker 42 stopped. I0423 21:01:34.371746 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 7.259292ms to process 0 records I0423 21:01:34.371879 1 tasks_processing.go:74] worker 21 stopped. I0423 21:01:34.371896 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 5.413145ms to process 0 records I0423 21:01:34.371917 1 tasks_processing.go:74] worker 14 stopped. I0423 21:01:34.372596 1 recorder.go:75] Recording config/infrastructure with fingerprint=261cbe73939524ea6642fe0778eef685515377cab250ee9a407f6f527d619322 I0423 21:01:34.372612 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 7.570742ms to process 1 records E0423 21:01:34.372622 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0423 21:01:34.372631 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 7.186162ms to process 0 records I0423 21:01:34.372637 1 tasks_processing.go:74] worker 18 stopped. E0423 21:01:34.374089 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:42611->172.30.0.10:53: read: connection refused I0423 21:01:34.374107 1 tasks_processing.go:74] worker 34 stopped. E0423 21:01:34.374113 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:42611->172.30.0.10:53: read: connection refused I0423 21:01:34.374125 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 9.556722ms to process 0 records E0423 21:01:34.374182 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:42611->172.30.0.10:53: read: connection refused I0423 21:01:34.374208 1 tasks_processing.go:74] worker 54 stopped. E0423 21:01:34.374215 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:37900->172.30.0.10:53: read: connection refused E0423 21:01:34.374220 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:42611->172.30.0.10:53: read: connection refused E0423 21:01:34.374221 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:37900->172.30.0.10:53: read: connection refused I0423 21:01:34.374246 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 9.73943ms to process 0 records E0423 21:01:34.374257 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:37900->172.30.0.10:53: read: connection refused I0423 21:01:34.374264 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 6.005494ms to process 0 records E0423 21:01:34.374271 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.15:37900->172.30.0.10:53: read: connection refused I0423 21:01:34.374278 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 8.197993ms to process 0 records I0423 21:01:34.374285 1 tasks_processing.go:74] worker 15 stopped. I0423 21:01:34.374289 1 tasks_processing.go:74] worker 47 stopped. I0423 21:01:34.375728 1 tasks_processing.go:74] worker 59 stopped. I0423 21:01:34.375828 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=cd864b3b787fa0c263a4fd0b1e8039d5d380c9519154c9b056e2cab8624e69e5 I0423 21:01:34.375865 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 9.969668ms to process 1 records I0423 21:01:34.388907 1 tasks_processing.go:74] worker 8 stopped. I0423 21:01:34.388921 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 24.208343ms to process 0 records I0423 21:01:34.388939 1 tasks_processing.go:74] worker 10 stopped. I0423 21:01:34.388948 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 21.731281ms to process 0 records E0423 21:01:34.388955 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0423 21:01:34.388961 1 gather.go:177] gatherer "clusterconfig" function "machines" took 21.474558ms to process 0 records I0423 21:01:34.388967 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 22.302529ms to process 0 records I0423 21:01:34.388971 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 24.274434ms to process 0 records I0423 21:01:34.388975 1 tasks_processing.go:74] worker 56 stopped. I0423 21:01:34.388978 1 tasks_processing.go:74] worker 7 stopped. I0423 21:01:34.388980 1 tasks_processing.go:74] worker 37 stopped. I0423 21:01:34.388981 1 tasks_processing.go:74] worker 30 stopped. I0423 21:01:34.388992 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 24.163328ms to process 0 records I0423 21:01:34.389001 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 24.244053ms to process 0 records I0423 21:01:34.389016 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 24.344869ms to process 0 records I0423 21:01:34.389021 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 20.670449ms to process 0 records I0423 21:01:34.389024 1 tasks_processing.go:74] worker 38 stopped. I0423 21:01:34.389026 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 24.377491ms to process 0 records I0423 21:01:34.389031 1 tasks_processing.go:74] worker 20 stopped. I0423 21:01:34.389032 1 tasks_processing.go:74] worker 2 stopped. I0423 21:01:34.389036 1 tasks_processing.go:74] worker 28 stopped. I0423 21:01:34.389041 1 tasks_processing.go:74] worker 33 stopped. I0423 21:01:34.389036 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 20.972588ms to process 0 records I0423 21:01:34.389127 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=aa12fedfeb4e3c01b1769366d0fce85cb38765ba1d53ba870e1295449518d3e7 I0423 21:01:34.389144 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=502d25a4ed416d35119068e131a8b81ec68405f9b07fc2d8e627faf275fb2f93 I0423 21:01:34.389152 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 22.925588ms to process 2 records I0423 21:01:34.389158 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 20.747474ms to process 0 records I0423 21:01:34.389163 1 tasks_processing.go:74] worker 43 stopped. I0423 21:01:34.389163 1 tasks_processing.go:74] worker 13 stopped. I0423 21:01:34.389260 1 tasks_processing.go:74] worker 3 stopped. E0423 21:01:34.389271 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0423 21:01:34.389278 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 24.879057ms to process 0 records I0423 21:01:34.389401 1 tasks_processing.go:74] worker 24 stopped. I0423 21:01:34.389457 1 recorder.go:75] Recording config/ingress with fingerprint=6d05433f68ff1d6481931509d13e46474b2b3fa557e74bc6eb305ed0756f1c22 I0423 21:01:34.389470 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 24.533481ms to process 1 records I0423 21:01:34.389482 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 21.044261ms to process 0 records I0423 21:01:34.389494 1 tasks_processing.go:74] worker 23 stopped. I0423 21:01:34.389565 1 tasks_processing.go:74] worker 35 stopped. I0423 21:01:34.389665 1 recorder.go:75] Recording config/image with fingerprint=f36959bb825c8ea99a38e0c13e20e3636a3a9660dd101577315128bbe49e6674 I0423 21:01:34.389683 1 gather.go:177] gatherer "clusterconfig" function "image" took 21.073866ms to process 1 records I0423 21:01:34.389986 1 tasks_processing.go:74] worker 46 stopped. I0423 21:01:34.390588 1 recorder.go:75] Recording config/oauth with fingerprint=e01aa4c8cd725b2ddd5e9aeee707da45ee3475b9ac1843e59227233afef84fe4 I0423 21:01:34.390651 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 25.288952ms to process 1 records I0423 21:01:34.397241 1 tasks_processing.go:74] worker 49 stopped. I0423 21:01:34.397261 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 28.955188ms to process 0 records I0423 21:01:34.397270 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 28.856857ms to process 0 records I0423 21:01:34.397277 1 tasks_processing.go:74] worker 45 stopped. I0423 21:01:34.397287 1 tasks_processing.go:74] worker 55 stopped. I0423 21:01:34.397298 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 31.421541ms to process 0 records I0423 21:01:34.397457 1 tasks_processing.go:74] worker 53 stopped. I0423 21:01:34.397567 1 recorder.go:75] Recording config/apiserver with fingerprint=1712650b84b3c02d1eb92d9759ef368e8f93229233ed564493670558c17ed79c I0423 21:01:34.397579 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 29.855193ms to process 1 records I0423 21:01:34.397727 1 tasks_processing.go:74] worker 4 stopped. I0423 21:01:34.397732 1 recorder.go:75] Recording config/authentication with fingerprint=6eec373764d2fef0108129d2b79745c37b9b1c982c4e010443b882b49e1da60d I0423 21:01:34.397747 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 32.88531ms to process 1 records I0423 21:01:34.397777 1 tasks_processing.go:74] worker 32 stopped. I0423 21:01:34.397782 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 29.743844ms to process 0 records I0423 21:01:34.398013 1 tasks_processing.go:74] worker 40 stopped. I0423 21:01:34.398028 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 32.696139ms to process 0 records I0423 21:01:34.398127 1 tasks_processing.go:74] worker 51 stopped. I0423 21:01:34.398142 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 30.062168ms to process 0 records I0423 21:01:34.398160 1 tasks_processing.go:74] worker 17 stopped. I0423 21:01:34.398234 1 recorder.go:75] Recording config/proxy with fingerprint=f05c3dbe2d5f96c32f3908814e3fb94c1fb57c2a0889b9029c70710401d06812 I0423 21:01:34.398247 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 31.989249ms to process 1 records I0423 21:01:34.398475 1 tasks_processing.go:74] worker 12 stopped. I0423 21:01:34.398597 1 recorder.go:75] Recording config/network with fingerprint=887b832d72efabb92c1d5cfb1a058381128eb49c9b4867eb9808e8c77ae66fd0 I0423 21:01:34.398609 1 gather.go:177] gatherer "clusterconfig" function "networks" took 30.456086ms to process 1 records I0423 21:01:34.401308 1 tasks_processing.go:74] worker 48 stopped. I0423 21:01:34.401461 1 recorder.go:75] Recording config/featuregate with fingerprint=76d2637cc5c795277554ca141013d68c7738c98df8351d192e7b4dbfa8fee3c7 I0423 21:01:34.401481 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 33.008271ms to process 1 records I0423 21:01:34.401667 1 tasks_processing.go:74] worker 26 stopped. I0423 21:01:34.401939 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=365c41ec86723f23671378c27c2d003b0859e5cfefb3c447d1fe936a0ffed955 I0423 21:01:34.401955 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 35.786738ms to process 1 records I0423 21:01:34.402044 1 tasks_processing.go:74] worker 31 stopped. I0423 21:01:34.402526 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=4db7dad92db61cd1530b837f6c74c2576fb9027d5c978a15049f5ad1b129a81f I0423 21:01:34.402539 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 33.777927ms to process 1 records I0423 21:01:34.402572 1 tasks_processing.go:74] worker 0 stopped. I0423 21:01:34.403384 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=b8c400fb9b2c280c67bf21ed5632fc1d1565f8f2dbe7070b16a247d4fa9daa73 I0423 21:01:34.403479 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=0c98a5cb49de95bbaeaa7520647d9a625990a0124f4a5f31b03b08c6b16090f0 I0423 21:01:34.403537 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=dc41191f586985d63660ecd46f413e0107a2eb4f38d433f426cfa64546cf60cb I0423 21:01:34.403552 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 33.711175ms to process 3 records I0423 21:01:34.403563 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0423 21:01:34.403581 1 controller.go:212] Source scaController *sca.Controller is not ready I0423 21:01:34.403586 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0423 21:01:34.403591 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 21:01:34.403594 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0423 21:01:34.403638 1 controller.go:489] The operator is still being initialized I0423 21:01:34.403648 1 controller.go:512] The operator is healthy I0423 21:01:34.405424 1 tasks_processing.go:74] worker 44 stopped. I0423 21:01:34.405712 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=36dddae68c8ecde293bc223b30979a1ca3cd20698e29b6110db6bcdcefeb5e78 I0423 21:01:34.405766 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=b1be84e17b6a3aef43c91331c19b4b0b3beb5101ae9aa7956498ac37bd6db721 I0423 21:01:34.405836 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=aa563e149fb83fcda4ab738466fb79a365966d341d85fe867797c5cf17bd7a99 I0423 21:01:34.405884 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=59a2a74b1a55c579df4bed719858e8ea7aeaaef37206c985d8ed5352db66b815 I0423 21:01:34.405928 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=e5bc3d01ba85986c6111f2569acf4de976809844947efe49359ad7ad5c3c6cb0 I0423 21:01:34.405974 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=02591dfd9d8a69c173600924a158aec61ff6ab69d4828b24052a284aadb1e3c8 I0423 21:01:34.406005 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=ba837f23b3e053f0c81238b77abfe1984de17f669ca76cf475d285d12334407c I0423 21:01:34.406660 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=1863657eb5d1118a9103c6982f2d0aac72cae39dde5b08bab22158718b4dc6a2 I0423 21:01:34.406706 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=468fa98a7ee2e150f163a2396973ba39aa1f20cd5ffc1ad322757d974062d9ff I0423 21:01:34.406740 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=1de3709cc37564b6aebcb51041f9c6adfd720e4a7ccc7f0ef2ea4981f7d5a684 I0423 21:01:34.406769 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=53f0b59d02002fe522f494b00f328c980a7d9b3b73cc653decaff2fef3e99398 I0423 21:01:34.406782 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 36.987872ms to process 11 records I0423 21:01:34.406826 1 tasks_processing.go:74] worker 61 stopped. I0423 21:01:34.406845 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 41.240163ms to process 0 records I0423 21:01:34.406897 1 tasks_processing.go:74] worker 39 stopped. I0423 21:01:34.407034 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=5b9e4b4871b6f7ef89974e1e356c77a6a53d565657464f2d062a0d88ca4dde90 I0423 21:01:34.407146 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=1f7c17c58554d8fb9250af1ed443f90d4d347ccc471c8cecb80e41c1f4fc94ef I0423 21:01:34.407280 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=bf42999d10a12c155635ea11d19da33fdaded814a6d9b6e53ec4acefff1d3ae9 I0423 21:01:34.407326 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=6082d34c97e4364aa28ccdda4a1392e181a68ae88b78bb1cf7f7f7cf5757e99c I0423 21:01:34.407363 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=7dc95271dec12b826fc0e4d88d16f46ef6ba6b89849d22888d0ce825c6bc607a I0423 21:01:34.407427 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=fe8224d582b8bb974dbdc21a7b0edbe5c627072100a31c44a58cc7ac8447b13b I0423 21:01:34.407469 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=0b2b019c89b4032ff3e7caee91b6df8e8b7b878bb6177b41429133b1ff4ba966 I0423 21:01:34.407506 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=93d093978951276e32838eba32a4ff82eb89acf0e29b85ca7a22ea49f0a7d576 I0423 21:01:34.407545 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=0f9ffd51d0cd618b08e69056e8e4cf7fb70e5e22b982bb9dbc707c0fdd565fc0 I0423 21:01:34.407608 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=af238762571d58dbf9649e0116bce4eb9f4d7e4f417a4de73bda357465b91c83 I0423 21:01:34.407647 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=603ac6597d619068121e1af47a576120dc3c1a09cbf9897890cf9b8529533768 I0423 21:01:34.407690 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=d3d69a844af234d62d39869c0023b499433900b4e801fa515ba45c8e7aae8e74 I0423 21:01:34.407750 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=e09aca962e689867e3e679642707d45c1c76c42ccce2ed0c5926a2b615139d5d I0423 21:01:34.407766 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 38.484003ms to process 13 records W0423 21:01:34.407811 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 21:01:34.407854 1 recorder.go:75] Recording config/olm_operators with fingerprint=adcd3f65705c29a217d14d01f0eb1e6d9c74cdc9c795e95db503bb5d8c98933d I0423 21:01:34.407868 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 40.769707ms to process 1 records I0423 21:01:34.407882 1 tasks_processing.go:74] worker 62 stopped. I0423 21:01:34.407947 1 tasks_processing.go:74] worker 50 stopped. I0423 21:01:34.408179 1 recorder.go:75] Recording config/node/ip-10-0-0-174.ec2.internal with fingerprint=1f62da6b9d0a4d6046f85510342f1c1c693034df7b7f2cdb37abc5f2ffe6a15e I0423 21:01:34.408246 1 recorder.go:75] Recording config/node/ip-10-0-1-80.ec2.internal with fingerprint=26630cc9d7f50f266922479259f9d9f1780bedc9965cc9a79e466ca885e3eaca I0423 21:01:34.408299 1 recorder.go:75] Recording config/node/ip-10-0-2-179.ec2.internal with fingerprint=15be246e5f3c3a4eeb2dff396945a905f36166b78ca4e8cec680e99f011e3c28 I0423 21:01:34.408305 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 40.917562ms to process 3 records I0423 21:01:34.408406 1 tasks_processing.go:74] worker 27 stopped. I0423 21:01:34.408756 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=b5299c946d54f2a2e8eaeca1bc308b13f5c21351fc3fed2bb4b8e47722ae672c I0423 21:01:34.408963 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=33d7c8760ff7e04b03dbe34b22df9905594076ea731be018085078c27d4390de I0423 21:01:34.408971 1 gather.go:177] gatherer "clusterconfig" function "crds" took 42.475268ms to process 2 records I0423 21:01:34.408985 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0423 21:01:34.408992 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 39.504079ms to process 1 records I0423 21:01:34.409051 1 tasks_processing.go:74] worker 58 stopped. I0423 21:01:34.409120 1 tasks_processing.go:74] worker 57 stopped. I0423 21:01:34.409134 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=c5e025a247e2e7c5219c3a95b5c03ed980386019e02b74d1818e801cc2b6d0bf I0423 21:01:34.409216 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=818bbe1d39adb0791a231adc44c6290bd774ed3321e995e61305edc30c41fbbf I0423 21:01:34.409225 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 39.665341ms to process 2 records I0423 21:01:34.412753 1 tasks_processing.go:74] worker 63 stopped. I0423 21:01:34.412777 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0423 21:01:34.412792 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 46.584756ms to process 1 records E0423 21:01:34.413187 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2767cd5ead-9715-4ee2-b684-66811226e9c6%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:56308->172.30.0.10:53: read: connection refused I0423 21:01:34.413200 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2767cd5ead-9715-4ee2-b684-66811226e9c6%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:56308->172.30.0.10:53: read: connection refused I0423 21:01:34.416362 1 tasks_processing.go:74] worker 25 stopped. I0423 21:01:34.416419 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 51.595651ms to process 0 records I0423 21:01:34.418943 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0423 21:01:34.422359 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:35978->172.30.0.10:53: read: connection refused I0423 21:01:34.422372 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:35978->172.30.0.10:53: read: connection refused I0423 21:01:34.425021 1 tasks_processing.go:74] worker 16 stopped. E0423 21:01:34.425034 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0423 21:01:34.425040 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ps35nrp0pk3bg30enkfd1917alf83qv-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ps35nrp0pk3bg30enkfd1917alf83qv-primary-cert-bundle-secret" not found I0423 21:01:34.425112 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=b5d460412fda13dc6f496730e0da71d609cd2e1bb4dd7fe93674c4bf342d743a I0423 21:01:34.425124 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 60.534194ms to process 1 records I0423 21:01:34.428882 1 tasks_processing.go:74] worker 5 stopped. E0423 21:01:34.428895 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0423 21:01:34.428901 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0423 21:01:34.428904 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0423 21:01:34.428913 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0423 21:01:34.428936 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0423 21:01:34.428950 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0423 21:01:34.428955 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0423 21:01:34.428963 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0423 21:01:34.429012 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0423 21:01:34.429019 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0423 21:01:34.429024 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 63.016068ms to process 7 records I0423 21:01:34.429042 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 21:01:34.429053 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0423 21:01:34.429070 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 21:01:34.434798 1 base_controller.go:82] Caches are synced for ConfigController I0423 21:01:34.434812 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0423 21:01:34.439092 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 21:01:34.454667 1 tasks_processing.go:74] worker 29 stopped. I0423 21:01:34.454672 1 gather_logs.go:145] no pods in namespace were found I0423 21:01:34.454924 1 recorder.go:75] Recording config/version with fingerprint=994f03860a7bbb68405ffce1d124f42a150b13ade734cd9d6a409c3c9dc198bf I0423 21:01:34.454938 1 recorder.go:75] Recording config/id with fingerprint=322a587684322e3367b19ebc11238c39b48ddb93c6b357b68b1c4b4f0ce2aa0e I0423 21:01:34.454944 1 gather.go:177] gatherer "clusterconfig" function "version" took 86.616281ms to process 2 records I0423 21:01:34.454955 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 86.643227ms to process 0 records I0423 21:01:34.454962 1 tasks_processing.go:74] worker 52 stopped. I0423 21:01:34.471954 1 base_controller.go:82] Caches are synced for LoggingSyncer I0423 21:01:34.471974 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0423 21:01:34.474694 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0423 21:01:34.478887 1 tasks_processing.go:74] worker 41 stopped. I0423 21:01:34.480129 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-94cdq with fingerprint=6045ece7068c7200cce57afc374b86519d9861b2ecf69362d474675c81c29888 I0423 21:01:34.480226 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-0-174.ec2.internal with fingerprint=4085cb974479f0d2492242ea4fc66ab7d99f930c76bff051a9e8924a316e8086 I0423 21:01:34.480296 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-1-80.ec2.internal with fingerprint=26d7524f3540735dcc78a06275921f01c5256a4586e07bd5db4f9c6cfdf2c5f9 I0423 21:01:34.480362 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-2-179.ec2.internal with fingerprint=86a80cf45c1bb47fda322b0a87a0fd01b3b79af893da0ea689a488039db89a95 I0423 21:01:34.480477 1 recorder.go:75] Recording config/pod/openshift-multus/multus-vvf7k with fingerprint=9d436e1bdeb9d10809bbf19db04d83f027265d5cd4f54807e858de28777a1442 I0423 21:01:34.480543 1 recorder.go:75] Recording config/running_containers with fingerprint=362ecc92fb25ff61e833f624310a51957b99dabceda48083bb6c42eb27fea594 I0423 21:01:34.480553 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 110.829151ms to process 6 records I0423 21:01:34.480565 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 114.163363ms to process 0 records I0423 21:01:34.480571 1 tasks_processing.go:74] worker 22 stopped. W0423 21:01:35.404904 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 21:01:36.225521 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0423 21:01:36.404646 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 21:01:36.412801 1 gather_cluster_operator_pods_and_events.go:121] Found 45 pods with 93 containers I0423 21:01:36.412813 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 270600 bytes I0423 21:01:36.412864 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:36.620682 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:36.819805 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:36.828560 1 tasks_processing.go:74] worker 9 stopped. I0423 21:01:36.828671 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=e39e6ab9dc4797c3c713c9a4fc6a8772946660f99e690f38f770652f15616ac7 I0423 21:01:36.828717 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0423 21:01:36.828741 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=d29e9bb40154e4650537b168e8295e45d39a9f5683ea36fe747281d579980ee3 I0423 21:01:36.828752 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0423 21:01:36.828774 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=09a4c4152974d08ae008a132103ea695c393c6eb1e656f95c2ed2ec1ca49a372 I0423 21:01:36.828789 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0423 21:01:36.828812 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=6d57be92250aa5b3db6cb2677c7f7aff0096bfb8171401db1b5e248b5a1e95a8 I0423 21:01:36.828842 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=f80c707f3a2cf41cbf99ef8b1dfbaf515d9cb0feb8f046b141f8ad1f13881d35 I0423 21:01:36.828865 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=3c26264bba33910d98f57785a9364cd33ddaa124464f7a9b129b41c1a90f1f94 I0423 21:01:36.828895 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=a51fedbdbf116ffc2ce0b01f7534a5bdfde4e2a041c7022c282422efb8169bad I0423 21:01:36.829142 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=8f3745dc46146f3d4fbcc916eb4ffe6689a7497ef962aced0d787d4fb8f7ea5a I0423 21:01:36.829162 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=7cd5bbbbe78c64a15e768b78ec9106f3ea910177c8fe28541e56d5d4f3e77bd3 I0423 21:01:36.829172 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0423 21:01:36.829188 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=317cbd307aed7d20d288f5b63765e5a2a475b314404bdb1699ac14172187df19 I0423 21:01:36.829198 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0423 21:01:36.829214 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=c0f48cda579761cc0ad4d332da1a3cf1373dfc111b4705f3334b97c68796d817 I0423 21:01:36.829224 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0423 21:01:36.829243 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=563ecc578388a50392ae7e66555ab085d22447374da69160b872a4b5d3434c94 I0423 21:01:36.829253 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0423 21:01:36.829272 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=cce12bc1227c5bb05cffbb19c2843fd4df8bc7fd57a34957ef1fb3d89dabceb6 I0423 21:01:36.829379 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=2e4205f75d23e2ecfc6cacd12566cc819e4b3b59069363696407a9347feb9c48 I0423 21:01:36.829405 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0423 21:01:36.829413 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0423 21:01:36.829434 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0423 21:01:36.829457 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=36e4e6aed98f5290381a0b493c32fae8be9fb4c3dec46f5f69aea3c02ac7d243 I0423 21:01:36.829483 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=b6bbf78d08c719a2572c696ab2abbef9a2f5b0dfe2efa5569646836d0118cb93 I0423 21:01:36.829493 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0423 21:01:36.829508 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=edc58b7fe169543f15349faf3563a2f45df212fa4c73f4158cfb2f33ffc0931c I0423 21:01:36.829517 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0423 21:01:36.829529 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=5159c8642e5bc93184b441fef97910f2cdcc8364c9c3ef125db5242ea4c3c01f I0423 21:01:36.829543 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=c13275093a48e6cb3ec7b107f2ce128fec3eea0ae2e05389ac30039f8b466242 I0423 21:01:36.829559 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=e4a59e4c1d0a82335d1aa482fd263acbef4b257634b80495e9538e6a9055a48a I0423 21:01:36.829575 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=408c791db32027c66220cc206df30430dff5742ebc5a6aa448bba8a312c5916a I0423 21:01:36.829596 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=6f7e034d0d368f44e07fddd5cb23690c25c62aa43930969e1f4055464b5d54f8 I0423 21:01:36.829605 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0423 21:01:36.829633 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=aabbe73285f2e2877973a515ceb669de658e5f3c780933276dc62148cf813170 I0423 21:01:36.829648 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0423 21:01:36.829657 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0423 21:01:36.829664 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.462472956s to process 38 records I0423 21:01:37.019024 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:37.217745 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). W0423 21:01:37.404623 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 21:01:37.418951 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:37.619510 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:37.817848 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:38.017278 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:38.217431 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:38.412259 1 tasks_processing.go:74] worker 11 stopped. I0423 21:01:38.412281 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=108780cb24f15bf572a4031966507398a0d0c6a28b3a008952f53b481dc191a9 I0423 21:01:38.412292 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 4.046118046s to process 1 records I0423 21:01:38.417266 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:38.617860 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:38.818514 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:39.017452 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:39.217877 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-monitoring-operator container cluster-monitoring-operator-698978dbbf-4zp5p pod in namespace openshift-monitoring (previous: false). I0423 21:01:39.420148 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-state-metrics container kube-state-metrics-7c87d874d5-6x67k pod in namespace openshift-monitoring (previous: false). I0423 21:01:39.618288 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container kube-state-metrics-7c87d874d5-6x67k pod in namespace openshift-monitoring (previous: false). I0423 21:01:39.817612 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container kube-state-metrics-7c87d874d5-6x67k pod in namespace openshift-monitoring (previous: false). I0423 21:01:40.020501 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-76dbdccb94-w7pgb pod in namespace openshift-monitoring (previous: false). I0423 21:01:40.257710 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-76dbdccb94-wk77c pod in namespace openshift-monitoring (previous: false). I0423 21:01:40.426635 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-67788f859f-2kgpj pod in namespace openshift-monitoring (previous: false). I0423 21:01:40.618780 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-67788f859f-fwzvh pod in namespace openshift-monitoring (previous: false). I0423 21:01:40.822551 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-lj69f pod in namespace openshift-monitoring (previous: false). I0423 21:01:41.019076 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 21:01:41.019092 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-lj69f pod in namespace openshift-monitoring (previous: false). I0423 21:01:41.218849 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-lj69f pod in namespace openshift-monitoring (previous: false). I0423 21:01:41.418861 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-lq54g pod in namespace openshift-monitoring (previous: false). I0423 21:01:41.618661 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 21:01:41.618675 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-lq54g pod in namespace openshift-monitoring (previous: false). I0423 21:01:41.819582 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-lq54g pod in namespace openshift-monitoring (previous: false). I0423 21:01:42.017831 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-lrxj9 pod in namespace openshift-monitoring (previous: false). I0423 21:01:42.216334 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 21:01:42.216349 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-lrxj9 pod in namespace openshift-monitoring (previous: false). I0423 21:01:42.418954 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-lrxj9 pod in namespace openshift-monitoring (previous: false). I0423 21:01:42.616907 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container openshift-state-metrics-6cb79d4dbd-bt525 pod in namespace openshift-monitoring (previous: false). I0423 21:01:42.818220 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container openshift-state-metrics-6cb79d4dbd-bt525 pod in namespace openshift-monitoring (previous: false). I0423 21:01:43.019774 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for openshift-state-metrics container openshift-state-metrics-6cb79d4dbd-bt525 pod in namespace openshift-monitoring (previous: false). I0423 21:01:43.219794 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:43.420470 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:43.620647 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:43.818169 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:44.019058 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:44.219115 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:44.417845 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 21:01:44.618770 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:44.819210 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:45.021423 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:45.218654 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:45.419728 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:45.617778 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:45.819240 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 21:01:46.019067 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator container prometheus-operator-5df5b8bf6d-dg2l8 pod in namespace openshift-monitoring (previous: false). I0423 21:01:46.220611 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-operator-5df5b8bf6d-dg2l8 pod in namespace openshift-monitoring (previous: false). I0423 21:01:46.417197 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-68c48fd977-bnxgn pod in namespace openshift-monitoring (previous: false). I0423 21:01:46.617697 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-68c48fd977-w6n8s pod in namespace openshift-monitoring (previous: false). I0423 21:01:46.816983 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-6dc9485868-6ldvv pod in namespace openshift-monitoring (previous: false). I0423 21:01:47.018167 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-6dc9485868-6ldvv pod in namespace openshift-monitoring (previous: false). I0423 21:01:47.213149 1 tasks_processing.go:74] worker 36 stopped. I0423 21:01:47.213202 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0423 21:01:47.213219 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.847117091s to process 1 records I0423 21:01:47.217959 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-6dc9485868-6ldvv pod in namespace openshift-monitoring (previous: false). I0423 21:01:47.417618 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-6dc9485868-6ldvv pod in namespace openshift-monitoring (previous: false). I0423 21:01:47.617841 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-6dc9485868-6ldvv pod in namespace openshift-monitoring (previous: false). I0423 21:01:47.817781 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-6dc9485868-6ldvv pod in namespace openshift-monitoring (previous: false). I0423 21:01:47.976041 1 tasks_processing.go:74] worker 60 stopped. I0423 21:01:47.976352 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=f5738032f28aea743b389898fadee779c5cd56b79a04dcc121fb2c80f0e9fc94 I0423 21:01:47.976368 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.611318279s to process 1 records I0423 21:01:48.019804 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-6dc9485868-x85qv pod in namespace openshift-monitoring (previous: false). I0423 21:01:48.217910 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-6dc9485868-x85qv pod in namespace openshift-monitoring (previous: false). I0423 21:01:48.417889 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-6dc9485868-x85qv pod in namespace openshift-monitoring (previous: false). I0423 21:01:48.617346 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-6dc9485868-x85qv pod in namespace openshift-monitoring (previous: false). I0423 21:01:48.818499 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-6dc9485868-x85qv pod in namespace openshift-monitoring (previous: false). I0423 21:01:49.018465 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-6dc9485868-x85qv pod in namespace openshift-monitoring (previous: false). I0423 21:01:49.217602 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-9mhfv pod in namespace openshift-network-console (previous: false). I0423 21:01:49.418953 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-wd8pl pod in namespace openshift-network-console (previous: false). I0423 21:01:49.617969 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-94cdq pod in namespace openshift-console-operator (previous: true). I0423 21:01:49.818257 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-94cdq pod in namespace openshift-console-operator (previous: false). I0423 21:01:50.021576 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-859979fd98-p7wb2 pod in namespace openshift-console (previous: false). I0423 21:01:50.218454 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-d7855875f-28x7w pod in namespace openshift-console (previous: false). I0423 21:01:50.419421 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-d7855875f-2zkrz pod in namespace openshift-console (previous: false). I0423 21:01:50.618068 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-dk6hp pod in namespace openshift-console (previous: false). I0423 21:01:50.818558 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-hhz8t pod in namespace openshift-console (previous: false). I0423 21:01:51.018623 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-lnzk6 pod in namespace openshift-dns (previous: false). I0423 21:01:51.221762 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-lnzk6 pod in namespace openshift-dns (previous: false). I0423 21:01:51.418263 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-pkpc8 pod in namespace openshift-dns (previous: false). I0423 21:01:51.618971 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-pkpc8 pod in namespace openshift-dns (previous: false). I0423 21:01:51.821630 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-vhx5f pod in namespace openshift-dns (previous: false). I0423 21:01:52.018246 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-vhx5f pod in namespace openshift-dns (previous: false). I0423 21:01:52.218664 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-9msbw pod in namespace openshift-dns (previous: false). I0423 21:01:52.418021 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-k4clz pod in namespace openshift-dns (previous: false). I0423 21:01:52.618658 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-xrgvj pod in namespace openshift-dns (previous: false). I0423 21:01:52.821434 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 21:01:52.821476 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-747bbf7864-4qncn pod in namespace openshift-image-registry (previous: true). I0423 21:01:53.019095 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-747bbf7864-4qncn pod in namespace openshift-image-registry (previous: false). I0423 21:01:53.218609 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-dd568498b-2ptrg pod in namespace openshift-image-registry (previous: false). I0423 21:01:53.419289 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-dd568498b-qfjrw pod in namespace openshift-image-registry (previous: false). I0423 21:01:53.619850 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-jdzfm pod in namespace openshift-image-registry (previous: false). I0423 21:01:53.817358 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 21:01:53.817378 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-skc98 pod in namespace openshift-image-registry (previous: false). I0423 21:01:54.021976 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 21:01:54.021994 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-xt6kr pod in namespace openshift-image-registry (previous: false). I0423 21:01:54.219157 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6bdcb487b7-n5mc2 pod in namespace openshift-ingress (previous: false). I0423 21:01:54.424775 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7567b576b5-cgbx7 pod in namespace openshift-ingress (previous: false). I0423 21:01:54.620799 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7567b576b5-kvzp2 pod in namespace openshift-ingress (previous: false). I0423 21:01:54.830350 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-cj5gl pod in namespace openshift-ingress-canary (previous: false). I0423 21:01:55.017864 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-v74lx pod in namespace openshift-ingress-canary (previous: false). I0423 21:01:55.218940 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-xpqsg pod in namespace openshift-ingress-canary (previous: false). I0423 21:01:55.420337 1 tasks_processing.go:74] worker 6 stopped. I0423 21:01:55.421030 1 recorder.go:75] Recording events/openshift-monitoring with fingerprint=2640f435c61b813e39b9bddc8590ec565add11bc3be0831f6a5705269d6b0155 I0423 21:01:55.421076 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=85bd053adfc7b1c3c69bf807784048fe654a1ecd75223f2868591759368505b8 I0423 21:01:55.421188 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=39186bebaffbd30b99058f29611076a0e23d63d3eac7a797b1dcf86085390338 I0423 21:01:55.421277 1 recorder.go:75] Recording events/openshift-console with fingerprint=ab332a789c94b0480aca3ef44eee01a2b78e9f8037a33cc5374111561768e882 I0423 21:01:55.421300 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=ee0477ab03ccaf5ba82bc692ce68ee7c9f148fb6d53bd7f136ec66b523f13b1b I0423 21:01:55.421431 1 recorder.go:75] Recording events/openshift-dns with fingerprint=99799c468f48dffbeb0b0953865ad601ddf037ef9a41caa973bd43c91b365b89 I0423 21:01:55.421586 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=a897a57a082bd233828536dd84ed774c21b402c5047533ccef704f2f13ba1d7c I0423 21:01:55.421612 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=e5801fbadb1d638b2844ee94aaeff372b74be905d608f9d5ef09d169149c8680 I0423 21:01:55.421680 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=46c218412325141c9f07da37274a4ac23de8215895d0825d2605ca0b432ad108 I0423 21:01:55.421720 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=b638ad454c033d78b54f8c5a7d6281cd3883de662c4c6f4c3c596fd7d215f964 I0423 21:01:55.421729 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/init-config-reloader_current.log with fingerprint=3e2c968dd735ad8f403079693340a31af36ced7bbe92898a22b4d0a63dcec362 I0423 21:01:55.421749 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/alertmanager_current.log with fingerprint=3bce40ffed3d4568a4fcb405628201443cfdf59f02c8b866b11af792f23c84b2 I0423 21:01:55.421763 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/config-reloader_current.log with fingerprint=173d78fb02553612a68bc2c628c00d9823f2c1c29d36e817bc829ed0db2a0fe7 I0423 21:01:55.421770 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-web_current.log with fingerprint=c87744fc49cada3f3ac81bbb879107528b138059f8c9897fbd66b7dc0b831268 I0423 21:01:55.421776 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy_current.log with fingerprint=8648673a4f674d2c20e9a17a93be003f777103b8f0ab53950533277b4376442f I0423 21:01:55.421783 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-metric_current.log with fingerprint=4cc57f1a95c271b7ce14c10edcabd7971550a250739fa65f1645a2eccbc30323 I0423 21:01:55.421788 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/prom-label-proxy_current.log with fingerprint=b3fd720d2b4fb7566589ca7c4efbb2489155e898bc7612eb9e0c5b201e4fa3ab I0423 21:01:55.421794 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/init-config-reloader_current.log with fingerprint=423dd8bb0798b1993692d2b1d44826a9e5295f8303d0a9f661c73f556e7b02a7 I0423 21:01:55.421814 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/alertmanager_current.log with fingerprint=5beb9e2b5c085da6942554312c0b4fdbb95db6ae4395d083b7427873648b6ef4 I0423 21:01:55.421828 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/config-reloader_current.log with fingerprint=f298eebf39ca368a3f2a922b7eb8180cea831b59ce5c6b8479480d6c7fa53e53 I0423 21:01:55.421834 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-web_current.log with fingerprint=4211b257529f2a9d5558888f25b651b8c2d7043843ad52f92baec155b52d8d48 I0423 21:01:55.421840 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy_current.log with fingerprint=beb625cc8dc82bce173d892da81d18e478058a8f142f3ae4678918a2a1382600 I0423 21:01:55.421845 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-metric_current.log with fingerprint=0ea6b104c214598f06e784f1e30eda5f5e48ac37426163d0874b0deb7de8ae2a I0423 21:01:55.421849 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/prom-label-proxy_current.log with fingerprint=4584f6118c07810d58a05a7527895ec76b2ef8dc75cbfdb3a42e0e6e07737ff8 I0423 21:01:55.421981 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/cluster-monitoring-operator-698978dbbf-4zp5p/cluster-monitoring-operator_current.log with fingerprint=0529fbc0012a4408aee2ced8bf1f470cfc1c53372037e4ddc26748c8f4439d9d I0423 21:01:55.422003 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-6x67k/kube-state-metrics_current.log with fingerprint=5c3a0da9ac2b8b7d8db53ba41af8551c7af6d6e1cebfc5116a128ba4ac6a998b I0423 21:01:55.422015 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-6x67k/kube-rbac-proxy-main_current.log with fingerprint=3e6cd5e3c007b81bd849a0b75dc7eba9cce3d277db42b30d81396402e4c39360 I0423 21:01:55.422022 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-6x67k/kube-rbac-proxy-self_current.log with fingerprint=81673b54f84790c27f45738012cc3ec2b8ca5632756d66780c90cbb70691e4d2 I0423 21:01:55.422030 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-76dbdccb94-w7pgb/metrics-server_current.log with fingerprint=07e61f17182de779d2b6a25ead865e2a5065a6c785bcd60293ed0dc19fef6a63 I0423 21:01:55.422040 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-76dbdccb94-wk77c/metrics-server_current.log with fingerprint=beb31e4fa4c9bce61721dd7d079cbaabed6ceec083598d46f44d181d71448e27 I0423 21:01:55.422050 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-67788f859f-2kgpj/monitoring-plugin_current.log with fingerprint=c7e710fc400e29bb8ab958f9b37ea1768b8e409f3cbbf5141f727718cc6e6d60 I0423 21:01:55.422062 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-67788f859f-fwzvh/monitoring-plugin_current.log with fingerprint=48871f710f40e0f320bcfde9ce0a4ac8cd465080a625ddf9ad9500407752cad8 I0423 21:01:55.422089 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-lj69f/node-exporter_current.log with fingerprint=c20fa3ad01de434dd9584724f6054a2e8602fd1beff0763721709b793158a9bf I0423 21:01:55.422098 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-lj69f/kube-rbac-proxy_current.log with fingerprint=ac42e05e4b9940fd71e7c0794c3cfd2fd6132e656758722ffda3cbaf3880f42c I0423 21:01:55.422124 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-lq54g/node-exporter_current.log with fingerprint=d66d95d9d3884f2b008e894bdefea162a5f9513104020986f28614f349130e54 I0423 21:01:55.422132 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-lq54g/kube-rbac-proxy_current.log with fingerprint=19d48b956eee3ce45d96f4d48988f825d30cb863955e82b3b4ba9c46f1613607 I0423 21:01:55.422159 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-lrxj9/node-exporter_current.log with fingerprint=960002e65d9314819b69b428028f266c4a02b68bd650f64aa863d862daa42e36 I0423 21:01:55.422167 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-lrxj9/kube-rbac-proxy_current.log with fingerprint=fba109921b2001fbbf29c4b06d2608cabe5ac8a62fe33761a77281ba99b9d8d6 I0423 21:01:55.422173 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-bt525/kube-rbac-proxy-main_current.log with fingerprint=793794bc737db8f5c0f7d47ae06577cc25855c5da7cb2f35fd72bfa78417cb51 I0423 21:01:55.422179 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-bt525/kube-rbac-proxy-self_current.log with fingerprint=7d3daf0dc6c054fb2232975d042dfd814b388b3dea479967dc3097ea08cf3dba I0423 21:01:55.422189 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-bt525/openshift-state-metrics_current.log with fingerprint=40db547528f92336ecd0d50f2af844026dd888738851db013a09d623b30036a7 I0423 21:01:55.422197 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/init-config-reloader_current.log with fingerprint=e83317f5c311c0dd266135c6fe75813434d2fd6c3d23ae718cf5c6cfcac55684 I0423 21:01:55.422293 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/prometheus_current.log with fingerprint=6d172f0d0a8976e26f3a0c50a6915501224a47941b351aea9cb987912b871a1c I0423 21:01:55.422307 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/config-reloader_current.log with fingerprint=5a94a28fe4ed936b7a40ecd0155cfd3d7c6c2cd3fbc3b63576a3ba7e7eff0640 I0423 21:01:55.422319 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/thanos-sidecar_current.log with fingerprint=af783a8f0a0f2fd58f720f5791a9ffa59c5c0ece1d15f3050ee85d87524b0851 I0423 21:01:55.422327 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-web_current.log with fingerprint=80139b02ce454511710049d0f941d28062e7c1647743c5c819f4b8c7d4d2cf48 I0423 21:01:55.422333 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy_current.log with fingerprint=92964f147c3cb419e9418b12f9d8928d9c798ba1d8200cb8fecaba64a0208bc2 I0423 21:01:55.422339 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-thanos_current.log with fingerprint=172168279479f757aec1d180a024848a5e55719cf140d5c417df05b26260efa0 I0423 21:01:55.422345 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/init-config-reloader_current.log with fingerprint=3bb595da8c6916072970b199c85e74c64957970b430915e8f6adccd85cfe75ce I0423 21:01:55.422453 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/prometheus_current.log with fingerprint=6d51cb1969f064e4f327411b3bb9be3b02e1875473e99bc6448529c6e86393b0 I0423 21:01:55.422467 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/config-reloader_current.log with fingerprint=1e4ac79e4be74688611acfd603ae41ebde9a9933020937cc4d56c06927cd700b I0423 21:01:55.422478 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/thanos-sidecar_current.log with fingerprint=ac82e86a1c4c98696b23df000840b91ede2c99d13f73b3ef455464b0b635db2e I0423 21:01:55.422485 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-web_current.log with fingerprint=b4fd3e139a190645e118170ecf3bcc0a570b6c6268543965e12f1f3de0207a73 I0423 21:01:55.422493 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy_current.log with fingerprint=dca218ac3d3721ac015c0c3161096075ccd62151525840ebf016808256cc1eff I0423 21:01:55.422499 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-thanos_current.log with fingerprint=1b56e867b6178e3d0d3df836b0dd87d9a315cd769dc2e0842cb1210c60d171b7 I0423 21:01:55.422578 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-5df5b8bf6d-dg2l8/prometheus-operator_current.log with fingerprint=bfa68fa28234d908a4c9caf2753d9073161d73954c6ae2983b28f07d72e26edd I0423 21:01:55.422586 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-5df5b8bf6d-dg2l8/kube-rbac-proxy_current.log with fingerprint=4eafc0770f46a98f8e5a73d39ec06e1e5a69319062a8a16eb85c803a38680071 I0423 21:01:55.422593 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-68c48fd977-bnxgn/prometheus-operator-admission-webhook_current.log with fingerprint=1a04e776a4a3fb865efa14702afe31cae5c3a440806ca42bc5cdb37f9752b959 I0423 21:01:55.422600 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-68c48fd977-w6n8s/prometheus-operator-admission-webhook_current.log with fingerprint=e62bdbb880a02cf554a32258c8a07ccd6fb2f18db37db5b53093ab581c2e2847 I0423 21:01:55.422624 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-6ldvv/thanos-query_current.log with fingerprint=e06a3f77efc50a256a8bf0b50b8216ff2b3be5d552f9e110f0022ceaf50289c7 I0423 21:01:55.422640 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-6ldvv/kube-rbac-proxy-web_current.log with fingerprint=5cf436bd2e6688ee450960e419a216c702f1899254823e1366c85a60420da943 I0423 21:01:55.422647 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-6ldvv/kube-rbac-proxy_current.log with fingerprint=c2fe48b522d0654257195dc74e8aee95282899c0d2314ce5ba1a87f2ec73b7ff I0423 21:01:55.422651 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-6ldvv/prom-label-proxy_current.log with fingerprint=3b67a25b119337d6ef7d605acfc56467ca0f66c88b6bc4fd3ac82ebec7f82982 I0423 21:01:55.422656 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-6ldvv/kube-rbac-proxy-rules_current.log with fingerprint=66243badc2d334e89e7da7381d2b150c33ac40908ac2f8535d0c294207aa0239 I0423 21:01:55.422663 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-6ldvv/kube-rbac-proxy-metrics_current.log with fingerprint=e55f5b18d2e6e20a316b0a1f0a269feeaf908381705f5f027d087c0cc5632cb9 I0423 21:01:55.422686 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-x85qv/thanos-query_current.log with fingerprint=be92d682e4d4e6def2905f72b8dc353467d2ccfb2bb2dfa678fde1ed0f79fad6 I0423 21:01:55.422701 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-x85qv/kube-rbac-proxy-web_current.log with fingerprint=e3def7cf5eab3a706060eb36bc2c51fd3d60176411bb208fffe65221c09c7754 I0423 21:01:55.422707 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-x85qv/kube-rbac-proxy_current.log with fingerprint=e3502ef6ce04ad5d717c05343d96048634b69bb4d0a864a0ff8c6a9e7004a115 I0423 21:01:55.422712 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-x85qv/prom-label-proxy_current.log with fingerprint=29f38102ec5c18f1fab614ea09dcf27c1a7a6ae2897a4d95f046596b8a85f2b0 I0423 21:01:55.422717 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-x85qv/kube-rbac-proxy-rules_current.log with fingerprint=90d26828b976974b6e1ce897b1142a3ab2a5691897f6a1973f5086d6b751cdbc I0423 21:01:55.422723 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6dc9485868-x85qv/kube-rbac-proxy-metrics_current.log with fingerprint=132bdcaa75b4a4562067dfbc003670cc346529f653e7f3562635529e470f0622 I0423 21:01:55.422729 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-9mhfv/networking-console-plugin_current.log with fingerprint=90a6390d1578ebc6fb0172a8d2d89f88a889c817e0dd424c48da3e916f4b9bbe I0423 21:01:55.422734 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-wd8pl/networking-console-plugin_current.log with fingerprint=c09e923e71e536f6f849d6bec7b80113e0bf008d9ff86beaa79c0c8f1cf94899 I0423 21:01:55.422863 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-94cdq with fingerprint=6045ece7068c7200cce57afc374b86519d9861b2ecf69362d474675c81c29888 E0423 21:01:55.422880 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json" was already recorded and had the fingerprint "6045ece7068c7200cce57afc374b86519d9861b2ecf69362d474675c81c29888", overwriting with the record having fingerprint "6045ece7068c7200cce57afc374b86519d9861b2ecf69362d474675c81c29888" W0423 21:01:55.422895 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json" because of the warning: warning: the record with the same fingerprint "6045ece7068c7200cce57afc374b86519d9861b2ecf69362d474675c81c29888" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json" I0423 21:01:55.422905 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-94cdq/console-operator_previous.log with fingerprint=8189af2f1a785e604fc69f83c22c5a73fd5590f5968ace161153bf7a34afe326 I0423 21:01:55.423029 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-94cdq/console-operator_current.log with fingerprint=f54f2f67f29e59d511950dfe56fd8795a10baee20e473efaf107bd2cb063b4bf I0423 21:01:55.423043 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-859979fd98-p7wb2/console_current.log with fingerprint=f1eadd3b95ba370b226f25a8efd38204ebb1eaa2cccf272a0f6f02b7995cf407 I0423 21:01:55.423057 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-d7855875f-28x7w/console_current.log with fingerprint=98215e4e1a24dcfb2ce577b04f1f983c69645d7d1d41f40f062292d19c27aac9 I0423 21:01:55.423071 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-d7855875f-2zkrz/console_current.log with fingerprint=be4d3bda55a181feb3c6ad15d3db54d0a68911a3505d7e3473bd1eedc7005baf I0423 21:01:55.423076 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-dk6hp/download-server_current.log with fingerprint=37fc2c881b8455ded2336aee280190af6ac702ed0c0a10849e77ba395193aa0b I0423 21:01:55.423081 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-hhz8t/download-server_current.log with fingerprint=b3bca08b2c8f5b8a1c6fe08a6d1f5b4424663e5d6fd1796b0eca6af286d2675f I0423 21:01:55.423172 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-lnzk6 with fingerprint=19c91ea4b4358bb0ad3a351f56be544d9c95daa091e09934df1044d4e84044e1 I0423 21:01:55.423180 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-lnzk6/dns_current.log with fingerprint=3ab28230575712c352b69322e0ed5ccc87794c72a8e70595d18dddf07ff5aa8e I0423 21:01:55.423187 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-lnzk6/kube-rbac-proxy_current.log with fingerprint=d7f8ac5baf06ad59d68888caacf96f2f42d1f9bf990bf85ab5a1bd8bf4efdd72 I0423 21:01:55.423192 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-pkpc8/dns_current.log with fingerprint=3917048b6cbc91c1d117f4ee716e1b1839c36da6bc4f6520c35864d2057e1417 I0423 21:01:55.423198 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-pkpc8/kube-rbac-proxy_current.log with fingerprint=3935c0b198d2590418502994ce60ea09f89d22c04f4e54b9d3703cd000d313b3 I0423 21:01:55.423203 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-vhx5f/dns_current.log with fingerprint=a5c29be5c7b3bb258002276d07796b2e4d989845cc0655f8b06aa5c3aa691740 I0423 21:01:55.423209 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-vhx5f/kube-rbac-proxy_current.log with fingerprint=e298fed58e861e14432be114783a58ae822c3996ad99260ce4a485ec26bd2348 I0423 21:01:55.423213 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-9msbw/dns-node-resolver_current.log with fingerprint=dbbfc53a749b70c91e5276e50057b76abf814a7d8606a2740d0f697e9c63b269 I0423 21:01:55.423219 1 recorder.go:75] Recording config/pod/openshift-dns/logs/node-resolver-k4clz/dns-node-resolver_current.log with fingerprint=43a77c51f39dba54957c61692359e70309bd2deaf0f812c3217d5ca605541e76 I0423 21:01:55.423327 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-747bbf7864-4qncn with fingerprint=a49c6057f204af9245f94d1de82e0921c6d461e7080a5f9b114e27dcd4af324b I0423 21:01:55.423361 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-747bbf7864-4qncn/registry_previous.log with fingerprint=cd4a1ae52d52725e9bf04d955798cdc3ba768e34db77b1a4b0f110b649521ab0 I0423 21:01:55.423386 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-747bbf7864-4qncn/registry_current.log with fingerprint=da4657aaee486a4c1a1c176c38cb5534ba95b482f744e0ee2ae238249c89e831 I0423 21:01:55.423439 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-dd568498b-2ptrg/registry_current.log with fingerprint=dac4ec8d31720f411801aa6b3f5d9db1376d0498f3b97f53fe9fd4fdda8c559a I0423 21:01:55.423476 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-dd568498b-qfjrw/registry_current.log with fingerprint=e96d6cc98b39a3f33a65caf5f3c974181294c8d393a6f02433b1499e4d413f53 I0423 21:01:55.423483 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-xt6kr/node-ca_current.log with fingerprint=bb74ca7c8f7891d72516408ffebbce955f6983b61b447e7d241a5a83ccab6dc5 I0423 21:01:55.423511 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-6bdcb487b7-n5mc2/router_current.log with fingerprint=a0a667ae03c28061d28fccd1c0491e66ce04cf15310a02bcbb582ab8881af9d8 I0423 21:01:55.423559 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-7567b576b5-cgbx7/router_current.log with fingerprint=17bbe14b94cd463007a36ad75aed4ac2136519da7b82db9f8bb17d982baef07b I0423 21:01:55.423607 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-7567b576b5-kvzp2/router_current.log with fingerprint=681a377cecbe19e28e3b214d101b178a1996ec1e856d7920f4f5a97cc9d3bf08 I0423 21:01:55.423614 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-cj5gl/serve-healthcheck-canary_current.log with fingerprint=918db45466ac27c01475ca8087cb7fda97d95049696a9626d0efe45b6de8dacc I0423 21:01:55.423619 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-v74lx/serve-healthcheck-canary_current.log with fingerprint=2a7e9a256650ccf8e4decd985c9cb9946ee960a7912ab095d37baebd1a536f34 I0423 21:01:55.423622 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-xpqsg/serve-healthcheck-canary_current.log with fingerprint=d381ff0f9a58085c285bbfff6e4873e63b94b2f10aa4e92d321c288d8432ce2c I0423 21:01:55.423628 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 21.051909808s to process 102 records E0423 21:01:55.423698 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 21.059s with: function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"metrics\" failed with an error, function \"tsdb_status\" failed with an error, function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json\"" I0423 21:01:55.424805 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "metrics" failed with an error, function "tsdb_status" failed with an error, function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-94cdq.json" I0423 21:01:55.424819 1 periodic.go:209] Running workloads gatherer I0423 21:01:55.424836 1 tasks_processing.go:45] number of workers: 2 I0423 21:01:55.424846 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 21:01:55.424853 1 tasks_processing.go:71] worker 1 working on workload_info task. I0423 21:01:55.424857 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 21:01:55.424883 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0423 21:01:55.432310 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-80.ec2.internal... I0423 21:01:55.432318 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-179.ec2.internal... I0423 21:01:55.432330 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-174.ec2.internal... I0423 21:01:55.459406 1 tasks_processing.go:74] worker 0 stopped. I0423 21:01:55.459430 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 34.489768ms to process 0 records I0423 21:02:00.017442 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0423 21:02:00.045246 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0423 21:02:00.053868 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0423 21:02:00.062911 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (9ms) I0423 21:02:00.071578 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0423 21:02:00.080332 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (9ms) I0423 21:02:00.089288 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (9ms) I0423 21:02:00.097689 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (8ms) I0423 21:02:00.106319 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (9ms) I0423 21:02:00.115625 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (9ms) I0423 21:02:00.124856 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (9ms) I0423 21:02:00.133417 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (9ms) I0423 21:02:00.156050 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (23ms) I0423 21:02:00.257511 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0423 21:02:00.353723 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (96ms) I0423 21:02:00.455032 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (101ms) I0423 21:02:00.553470 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (98ms) I0423 21:02:00.654793 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (101ms) I0423 21:02:00.752898 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (98ms) I0423 21:02:00.854418 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (101ms) I0423 21:02:00.953650 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (99ms) I0423 21:02:01.054386 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0423 21:02:01.153783 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0423 21:02:01.255678 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (102ms) I0423 21:02:01.353511 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (98ms) I0423 21:02:01.454784 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (101ms) I0423 21:02:01.554492 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0423 21:02:01.656974 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (102ms) I0423 21:02:01.754238 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (97ms) I0423 21:02:01.854933 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (101ms) I0423 21:02:01.954014 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (99ms) I0423 21:02:02.054141 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (100ms) I0423 21:02:02.154445 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0423 21:02:02.254780 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (100ms) I0423 21:02:02.354797 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (100ms) I0423 21:02:02.455326 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (101ms) I0423 21:02:02.554845 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0423 21:02:02.654111 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (99ms) I0423 21:02:02.754546 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (100ms) I0423 21:02:02.854644 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0423 21:02:02.955249 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (101ms) I0423 21:02:03.054151 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (99ms) I0423 21:02:03.154349 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0423 21:02:03.254579 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (100ms) I0423 21:02:03.354755 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (100ms) I0423 21:02:03.455118 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (100ms) I0423 21:02:03.554549 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (99ms) I0423 21:02:03.654861 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0423 21:02:03.755051 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0423 21:02:03.854638 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0423 21:02:03.954968 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (100ms) I0423 21:02:03.955003 1 tasks_processing.go:74] worker 1 stopped. I0423 21:02:03.955327 1 recorder.go:75] Recording config/workload_info with fingerprint=c9cc5c27da3f23f67ecbd4d5edc7fa2a03d22c19b73395647e1992e63f1c3b8b I0423 21:02:03.955353 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.530142446s to process 1 records I0423 21:02:03.955373 1 periodic.go:256] Periodic gather workloads completed in 8.53s I0423 21:02:03.955384 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0423 21:02:03.955414 1 periodic.go:209] Running conditional gatherer I0423 21:02:03.962596 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0423 21:02:04.144415 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0423 21:02:05.214459 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0423 21:02:05.222488 1 conditional_gatherer.go:392] cluster version is '4.20.8' I0423 21:02:05.222521 1 tasks_processing.go:45] number of workers: 3 I0423 21:02:05.222541 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 21:02:05.222546 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0423 21:02:05.222556 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 21:02:05.222563 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 21:02:05.222576 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0423 21:02:05.222585 1 tasks_processing.go:74] worker 1 stopped. I0423 21:02:05.222579 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0423 21:02:05.222630 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0423 21:02:05.222643 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 880ns to process 1 records I0423 21:02:05.222677 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0423 21:02:05.222685 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.283µs to process 1 records I0423 21:02:05.222690 1 tasks_processing.go:74] worker 0 stopped. I0423 21:02:05.222867 1 tasks_processing.go:74] worker 2 stopped. I0423 21:02:05.222885 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 271.398µs to process 0 records I0423 21:02:05.222915 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0423 21:02:05.222933 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0423 21:02:05.251871 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=ddbe2848faa786bd85517ebd339e8413e1a25cb3810c509543a7d0ac93fc5758 I0423 21:02:05.252048 1 diskrecorder.go:70] Writing 213 records to /var/lib/insights-operator/insights-2026-04-23-210205.tar.gz I0423 21:02:05.267417 1 diskrecorder.go:51] Wrote 213 records to disk in 15ms I0423 21:02:05.267458 1 periodic.go:278] Gathering cluster info every 2h0m0s I0423 21:02:05.267476 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0423 21:03:04.365339 1 diskrecorder.go:223] Found files to send: insights-2026-04-23-210205.tar.gz I0423 21:03:04.365367 1 insightsuploader.go:150] Checking archives to upload periodically every 16m12.712779417s I0423 21:03:04.365376 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0423 21:03:04.376642 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0423 21:03:04.777778 1 requests.go:87] Successfully reported id=2026-04-23T21:03:04Z x-rh-insights-request-id=26a723702603476eabfa22424810503c, wrote=110852 I0423 21:03:04.777894 1 insightsuploader.go:187] Uploaded report successfully in 412.5082ms I0423 21:03:04.777920 1 controller.go:128] Initializing last reported time to 2026-04-23T21:03:04Z I0423 21:03:04.777997 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0423 21:03:04.778011 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0423 21:03:04.778021 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0423 21:03:04.783858 1 controller.go:512] The operator is healthy I0423 21:03:34.378800 1 controller.go:512] The operator is healthy I0423 21:04:05.494279 1 insightsreport.go:137] Pulling report from smart-proxy I0423 21:04:05.494303 1 insightsreport.go:149] Retrieving report I0423 21:04:05.501927 1 requests.go:110] Retrieving report for cluster: 67cd5ead-9715-4ee2-b684-66811226e9c6 I0423 21:04:05.501941 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/67cd5ead-9715-4ee2-b684-66811226e9c6/reports I0423 21:04:05.504804 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/67cd5ead-9715-4ee2-b684-66811226e9c6/reports I0423 21:04:05.685548 1 insightsreport.go:184] Report retrieved I0423 21:04:05.711917 1 insightsreport.go:239] Report retrieved correctly I0423 21:05:34.378491 1 controller.go:512] The operator is healthy