I0423 14:41:46.082392 1 cmd.go:253] Using service-serving-cert provided certificates I0423 14:41:46.082820 1 observer_polling.go:159] Starting file observer I0423 14:41:46.365604 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0423 14:41:46.365788 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0423 14:41:46.366343 1 secure_serving.go:57] Forcing use of http/1.1 only W0423 14:41:46.366363 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0423 14:41:46.366367 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0423 14:41:46.366371 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0423 14:41:46.366374 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0423 14:41:46.366376 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0423 14:41:46.366378 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0423 14:41:46.366494 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0423 14:41:46.370479 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0423 14:41:46.370495 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"c0bc65cd-4aa5-4387-bb3d-eb44ff02a2c7", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0423 14:41:46.373230 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0423 14:41:46.373242 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0423 14:41:46.373254 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0423 14:41:46.373270 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 14:41:46.373252 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0423 14:41:46.373282 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 14:41:46.374540 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0423 14:41:46.374795 1 secure_serving.go:213] Serving securely on [::]:8443 I0423 14:41:46.374822 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0423 14:41:46.378476 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0423 14:41:46.378501 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 14:41:46.378609 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0423 14:41:46.383813 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 14:41:46.383830 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0423 14:41:46.388761 1 secretconfigobserver.go:119] support secret does not exist I0423 14:41:46.393505 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 14:41:46.397726 1 secretconfigobserver.go:119] support secret does not exist I0423 14:41:46.400126 1 recorder.go:161] Pruning old reports every 6h48m20s, max age is 288h0m0s I0423 14:41:46.404778 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0423 14:41:46.404795 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0423 14:41:46.404798 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0423 14:41:46.404806 1 periodic.go:209] Running clusterconfig gatherer I0423 14:41:46.404845 1 tasks_processing.go:45] number of workers: 64 I0423 14:41:46.404886 1 tasks_processing.go:69] worker 63 listening for tasks. I0423 14:41:46.404910 1 tasks_processing.go:71] worker 63 working on machine_autoscalers task. I0423 14:41:46.404913 1 tasks_processing.go:69] worker 49 listening for tasks. I0423 14:41:46.404919 1 tasks_processing.go:69] worker 25 listening for tasks. I0423 14:41:46.404923 1 tasks_processing.go:69] worker 36 listening for tasks. I0423 14:41:46.404919 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 14:41:46.404927 1 tasks_processing.go:69] worker 56 listening for tasks. I0423 14:41:46.404886 1 tasks_processing.go:69] worker 35 listening for tasks. I0423 14:41:46.404807 1 insightsreport.go:296] Starting report retriever I0423 14:41:46.404942 1 tasks_processing.go:69] worker 4 listening for tasks. I0423 14:41:46.404942 1 tasks_processing.go:69] worker 27 listening for tasks. I0423 14:41:46.404926 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 14:41:46.404950 1 tasks_processing.go:69] worker 28 listening for tasks. I0423 14:41:46.404930 1 tasks_processing.go:69] worker 26 listening for tasks. I0423 14:41:46.404935 1 tasks_processing.go:69] worker 3 listening for tasks. I0423 14:41:46.404958 1 tasks_processing.go:69] worker 5 listening for tasks. I0423 14:41:46.404960 1 tasks_processing.go:69] worker 46 listening for tasks. I0423 14:41:46.404938 1 tasks_processing.go:69] worker 37 listening for tasks. I0423 14:41:46.404958 1 tasks_processing.go:69] worker 57 listening for tasks. I0423 14:41:46.404966 1 tasks_processing.go:69] worker 55 listening for tasks. I0423 14:41:46.404947 1 tasks_processing.go:69] worker 44 listening for tasks. I0423 14:41:46.404971 1 tasks_processing.go:69] worker 30 listening for tasks. I0423 14:41:46.404947 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0423 14:41:46.404974 1 tasks_processing.go:69] worker 48 listening for tasks. I0423 14:41:46.404976 1 tasks_processing.go:69] worker 59 listening for tasks. I0423 14:41:46.404982 1 tasks_processing.go:69] worker 31 listening for tasks. I0423 14:41:46.404983 1 tasks_processing.go:69] worker 42 listening for tasks. I0423 14:41:46.404987 1 tasks_processing.go:69] worker 61 listening for tasks. I0423 14:41:46.404988 1 tasks_processing.go:69] worker 52 listening for tasks. I0423 14:41:46.404993 1 tasks_processing.go:69] worker 33 listening for tasks. I0423 14:41:46.404995 1 tasks_processing.go:69] worker 17 listening for tasks. I0423 14:41:46.404886 1 tasks_processing.go:69] worker 23 listening for tasks. I0423 14:41:46.405000 1 tasks_processing.go:69] worker 18 listening for tasks. I0423 14:41:46.404997 1 tasks_processing.go:69] worker 62 listening for tasks. I0423 14:41:46.405004 1 tasks_processing.go:69] worker 19 listening for tasks. I0423 14:41:46.405008 1 tasks_processing.go:69] worker 12 listening for tasks. I0423 14:41:46.405012 1 tasks_processing.go:69] worker 22 listening for tasks. I0423 14:41:46.405011 1 tasks_processing.go:71] worker 36 working on machine_config_pools task. I0423 14:41:46.405016 1 tasks_processing.go:71] worker 46 working on machine_configs task. I0423 14:41:46.405017 1 tasks_processing.go:71] worker 30 working on overlapping_namespace_uids task. I0423 14:41:46.405018 1 tasks_processing.go:69] worker 13 listening for tasks. I0423 14:41:46.405023 1 tasks_processing.go:71] worker 61 working on openstack_dataplanedeployments task. I0423 14:41:46.405024 1 tasks_processing.go:71] worker 48 working on qemu_kubevirt_launcher_logs task. I0423 14:41:46.405022 1 tasks_processing.go:69] worker 10 listening for tasks. I0423 14:41:46.405031 1 tasks_processing.go:71] worker 1 working on image task. I0423 14:41:46.405034 1 tasks_processing.go:71] worker 12 working on validating_webhook_configurations task. I0423 14:41:46.405037 1 tasks_processing.go:71] worker 18 working on active_alerts task. I0423 14:41:46.405040 1 tasks_processing.go:71] worker 35 working on crds task. I0423 14:41:46.405041 1 tasks_processing.go:71] worker 56 working on proxies task. I0423 14:41:46.405043 1 tasks_processing.go:71] worker 4 working on nodenetworkstates task. I0423 14:41:46.405050 1 tasks_processing.go:71] worker 3 working on cost_management_metrics_configs task. I0423 14:41:46.405055 1 tasks_processing.go:71] worker 26 working on storage_classes task. I0423 14:41:46.405083 1 tasks_processing.go:71] worker 2 working on ceph_cluster task. I0423 14:41:46.405017 1 tasks_processing.go:71] worker 55 working on machine_healthchecks task. I0423 14:41:46.405383 1 tasks_processing.go:71] worker 28 working on cluster_apiserver task. I0423 14:41:46.405039 1 tasks_processing.go:71] worker 31 working on install_plans task. I0423 14:41:46.405007 1 tasks_processing.go:69] worker 11 listening for tasks. I0423 14:41:46.405033 1 tasks_processing.go:71] worker 62 working on service_accounts task. I0423 14:41:46.405567 1 tasks_processing.go:71] worker 11 working on container_runtime_configs task. I0423 14:41:46.404911 1 tasks_processing.go:69] worker 24 listening for tasks. I0423 14:41:46.405025 1 tasks_processing.go:71] worker 17 working on nodes task. I0423 14:41:46.405683 1 tasks_processing.go:71] worker 24 working on node_logs task. I0423 14:41:46.404999 1 tasks_processing.go:69] worker 34 listening for tasks. I0423 14:41:46.404992 1 tasks_processing.go:69] worker 32 listening for tasks. I0423 14:41:46.404911 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 14:41:46.404982 1 tasks_processing.go:69] worker 51 listening for tasks. I0423 14:41:46.405879 1 tasks_processing.go:71] worker 34 working on ingress_certificates task. I0423 14:41:46.405977 1 tasks_processing.go:71] worker 51 working on version task. I0423 14:41:46.406084 1 tasks_processing.go:71] worker 32 working on authentication task. I0423 14:41:46.406265 1 tasks_processing.go:71] worker 0 working on dvo_metrics task. I0423 14:41:46.404947 1 tasks_processing.go:69] worker 60 listening for tasks. I0423 14:41:46.406371 1 tasks_processing.go:71] worker 60 working on image_pruners task. I0423 14:41:46.404954 1 tasks_processing.go:69] worker 45 listening for tasks. I0423 14:41:46.404955 1 tasks_processing.go:69] worker 50 listening for tasks. I0423 14:41:46.404954 1 tasks_processing.go:69] worker 43 listening for tasks. I0423 14:41:46.404961 1 tasks_processing.go:69] worker 54 listening for tasks. I0423 14:41:46.404965 1 tasks_processing.go:69] worker 29 listening for tasks. I0423 14:41:46.404967 1 tasks_processing.go:69] worker 6 listening for tasks. I0423 14:41:46.404968 1 tasks_processing.go:69] worker 38 listening for tasks. I0423 14:41:46.404968 1 tasks_processing.go:69] worker 58 listening for tasks. I0423 14:41:46.404973 1 tasks_processing.go:69] worker 7 listening for tasks. I0423 14:41:46.404972 1 tasks_processing.go:69] worker 53 listening for tasks. I0423 14:41:46.404976 1 tasks_processing.go:69] worker 39 listening for tasks. I0423 14:41:46.404981 1 tasks_processing.go:69] worker 41 listening for tasks. I0423 14:41:46.404981 1 tasks_processing.go:69] worker 8 listening for tasks. I0423 14:41:46.404988 1 tasks_processing.go:69] worker 9 listening for tasks. I0423 14:41:46.404987 1 tasks_processing.go:69] worker 40 listening for tasks. I0423 14:41:46.404984 1 tasks_processing.go:69] worker 47 listening for tasks. I0423 14:41:46.404996 1 tasks_processing.go:69] worker 16 listening for tasks. I0423 14:41:46.404998 1 tasks_processing.go:69] worker 20 listening for tasks. I0423 14:41:46.404998 1 tasks_processing.go:69] worker 14 listening for tasks. I0423 14:41:46.405004 1 tasks_processing.go:69] worker 21 listening for tasks. I0423 14:41:46.405002 1 tasks_processing.go:69] worker 15 listening for tasks. I0423 14:41:46.405011 1 tasks_processing.go:71] worker 25 working on openstack_controlplanes task. I0423 14:41:46.405011 1 tasks_processing.go:71] worker 49 working on tsdb_status task. I0423 14:41:46.407368 1 tasks_processing.go:71] worker 54 working on openstack_version task. I0423 14:41:46.407439 1 tasks_processing.go:71] worker 16 working on monitoring_persistent_volumes task. I0423 14:41:46.407371 1 tasks_processing.go:71] worker 43 working on nodenetworkconfigurationpolicies task. I0423 14:41:46.405020 1 tasks_processing.go:71] worker 42 working on certificate_signing_requests task. I0423 14:41:46.405018 1 tasks_processing.go:71] worker 44 working on openshift_machine_api_events task. I0423 14:41:46.405019 1 tasks_processing.go:71] worker 33 working on image_registries task. I0423 14:41:46.407625 1 tasks_processing.go:71] worker 47 working on aggregated_monitoring_cr_names task. I0423 14:41:46.407764 1 tasks_processing.go:71] worker 38 working on schedulers task. I0423 14:41:46.407830 1 tasks_processing.go:71] worker 20 working on sap_pods task. I0423 14:41:46.405022 1 tasks_processing.go:71] worker 37 working on networks task. I0423 14:41:46.408102 1 tasks_processing.go:71] worker 40 working on jaegers task. I0423 14:41:46.408118 1 tasks_processing.go:71] worker 6 working on oauths task. I0423 14:41:46.408132 1 tasks_processing.go:71] worker 15 working on sap_config task. I0423 14:41:46.408129 1 tasks_processing.go:71] worker 58 working on operators_pods_and_events task. I0423 14:41:46.405026 1 tasks_processing.go:71] worker 27 working on machines task. I0423 14:41:46.407374 1 tasks_processing.go:71] worker 21 working on mutating_webhook_configurations task. I0423 14:41:46.408253 1 tasks_processing.go:71] worker 39 working on ingress task. I0423 14:41:46.405025 1 tasks_processing.go:71] worker 59 working on feature_gates task. I0423 14:41:46.405028 1 tasks_processing.go:71] worker 13 working on storage_cluster task. I0423 14:41:46.408264 1 tasks_processing.go:71] worker 9 working on openshift_logging task. I0423 14:41:46.405026 1 tasks_processing.go:71] worker 57 working on pod_network_connectivity_checks task. I0423 14:41:46.405030 1 tasks_processing.go:71] worker 23 working on machine_sets task. I0423 14:41:46.405034 1 tasks_processing.go:71] worker 19 working on pdbs task. I0423 14:41:46.405036 1 tasks_processing.go:71] worker 10 working on clusterroles task. I0423 14:41:46.405034 1 tasks_processing.go:71] worker 52 working on operators task. I0423 14:41:46.405100 1 tasks_processing.go:71] worker 5 working on lokistack task. I0423 14:41:46.407368 1 tasks_processing.go:71] worker 8 working on config_maps task. I0423 14:41:46.407450 1 tasks_processing.go:71] worker 14 working on openstack_dataplanenodesets task. I0423 14:41:46.407496 1 tasks_processing.go:71] worker 41 working on olm_operators task. I0423 14:41:46.405020 1 tasks_processing.go:71] worker 22 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0423 14:41:46.408192 1 tasks_processing.go:71] worker 45 working on infrastructures task. I0423 14:41:46.408210 1 tasks_processing.go:71] worker 29 working on container_images task. I0423 14:41:46.408220 1 tasks_processing.go:71] worker 53 working on support_secret task. I0423 14:41:46.408228 1 tasks_processing.go:71] worker 50 working on metrics task. I0423 14:41:46.408890 1 tasks_processing.go:71] worker 7 working on silenced_alerts task. I0423 14:41:46.411261 1 tasks_processing.go:71] worker 61 working on sap_datahubs task. I0423 14:41:46.411322 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 6.224819ms to process 0 records I0423 14:41:46.412049 1 tasks_processing.go:74] worker 63 stopped. I0423 14:41:46.412062 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 7.130016ms to process 0 records E0423 14:41:46.412809 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:53792->172.30.0.10:53: read: connection refused I0423 14:41:46.412824 1 tasks_processing.go:74] worker 7 stopped. E0423 14:41:46.412830 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:53792->172.30.0.10:53: read: connection refused I0423 14:41:46.412839 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 3.908282ms to process 0 records E0423 14:41:46.412852 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:53792->172.30.0.10:53: read: connection refused I0423 14:41:46.412868 1 tasks_processing.go:74] worker 18 stopped. E0423 14:41:46.412874 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:53792->172.30.0.10:53: read: connection refused I0423 14:41:46.412882 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 7.819929ms to process 0 records E0423 14:41:46.412960 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:38419->172.30.0.10:53: read: connection refused I0423 14:41:46.412976 1 tasks_processing.go:74] worker 49 stopped. E0423 14:41:46.412984 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:38419->172.30.0.10:53: read: connection refused E0423 14:41:46.412978 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:38419->172.30.0.10:53: read: connection refused I0423 14:41:46.412997 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 6.097537ms to process 0 records E0423 14:41:46.413019 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.10:38419->172.30.0.10:53: read: connection refused I0423 14:41:46.413037 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 4.435488ms to process 0 records I0423 14:41:46.413048 1 tasks_processing.go:74] worker 50 stopped. I0423 14:41:46.414368 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0423 14:41:46.414383 1 controller.go:326] The initial operator extension status is healthy I0423 14:41:46.414391 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 14:41:46.414395 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0423 14:41:46.414397 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0423 14:41:46.414416 1 controller.go:489] The operator is still being initialized I0423 14:41:46.414423 1 controller.go:512] The operator is healthy I0423 14:41:46.414487 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0423 14:41:46.414490 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0423 14:41:46.414990 1 operator.go:288] started I0423 14:41:46.415015 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0423 14:41:46.415030 1 tasks_processing.go:74] worker 56 stopped. I0423 14:41:46.415124 1 recorder.go:75] Recording config/proxy with fingerprint=832f7398874007fe38fd6892cdf811255d2b3c808feb5674e79451c9fae30194 I0423 14:41:46.415137 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 9.97845ms to process 1 records I0423 14:41:46.415255 1 tasks_processing.go:74] worker 28 stopped. I0423 14:41:46.415426 1 recorder.go:75] Recording config/apiserver with fingerprint=d86f4fe5254f597c076f879b0d642c50c8456ec1b01e1c839ad7b1c85537016e I0423 14:41:46.415443 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 9.850608ms to process 1 records I0423 14:41:46.418097 1 tasks_processing.go:74] worker 4 stopped. I0423 14:41:46.418112 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 13.040396ms to process 0 records I0423 14:41:46.418852 1 tasks_processing.go:74] worker 11 stopped. I0423 14:41:46.418862 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 13.27429ms to process 0 records I0423 14:41:46.420464 1 tasks_processing.go:74] worker 25 stopped. I0423 14:41:46.420472 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 13.894855ms to process 0 records I0423 14:41:46.420480 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 15.33174ms to process 0 records I0423 14:41:46.420485 1 tasks_processing.go:74] worker 2 stopped. I0423 14:41:46.420984 1 tasks_processing.go:74] worker 3 stopped. I0423 14:41:46.420996 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 15.924409ms to process 0 records I0423 14:41:46.421002 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 12.860515ms to process 0 records I0423 14:41:46.421007 1 tasks_processing.go:74] worker 15 stopped. I0423 14:41:46.421007 1 gather_logs.go:145] no pods in namespace were found I0423 14:41:46.421017 1 tasks_processing.go:74] worker 48 stopped. I0423 14:41:46.421024 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 15.983828ms to process 0 records I0423 14:41:46.421207 1 tasks_processing.go:74] worker 1 stopped. I0423 14:41:46.421292 1 recorder.go:75] Recording config/image with fingerprint=72f79be99e9d81bc527722c7678a9373f6f35d9341fb3539d8ab0958392ad043 I0423 14:41:46.421303 1 gather.go:177] gatherer "clusterconfig" function "image" took 16.161457ms to process 1 records I0423 14:41:46.422142 1 tasks_processing.go:74] worker 14 stopped. I0423 14:41:46.422150 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 13.761539ms to process 0 records I0423 14:41:46.425568 1 tasks_processing.go:74] worker 61 stopped. I0423 14:41:46.425583 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 14.294338ms to process 0 records I0423 14:41:46.425893 1 tasks_processing.go:74] worker 57 stopped. E0423 14:41:46.425942 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0423 14:41:46.425959 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 17.612267ms to process 0 records I0423 14:41:46.426107 1 tasks_processing.go:74] worker 55 stopped. E0423 14:41:46.426119 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0423 14:41:46.426125 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 20.766877ms to process 0 records I0423 14:41:46.435505 1 tasks_processing.go:74] worker 13 stopped. I0423 14:41:46.435519 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 27.233408ms to process 0 records I0423 14:41:46.435527 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 27.638704ms to process 0 records I0423 14:41:46.435533 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 28.073447ms to process 0 records I0423 14:41:46.435538 1 tasks_processing.go:74] worker 20 stopped. I0423 14:41:46.435539 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 27.26211ms to process 0 records I0423 14:41:46.435544 1 tasks_processing.go:74] worker 9 stopped. I0423 14:41:46.435539 1 tasks_processing.go:74] worker 43 stopped. E0423 14:41:46.435552 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0423 14:41:46.435558 1 tasks_processing.go:74] worker 53 stopped. I0423 14:41:46.435562 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 26.993262ms to process 0 records I0423 14:41:46.435943 1 tasks_processing.go:74] worker 23 stopped. I0423 14:41:46.435955 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 27.655439ms to process 0 records I0423 14:41:46.435987 1 tasks_processing.go:74] worker 54 stopped. I0423 14:41:46.435996 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 28.605061ms to process 0 records I0423 14:41:46.436044 1 tasks_processing.go:74] worker 45 stopped. I0423 14:41:46.436049 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0423 14:41:46.436072 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0423 14:41:46.436077 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0423 14:41:46.436082 1 controller.go:212] Source scaController *sca.Controller is not ready I0423 14:41:46.436088 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0423 14:41:46.436108 1 controller.go:489] The operator is still being initialized I0423 14:41:46.436120 1 controller.go:512] The operator is healthy I0423 14:41:46.436641 1 recorder.go:75] Recording config/infrastructure with fingerprint=a7fa03692a1eef6399141645eb66f1e81b3992a4eff1468ad82eed76df462d90 I0423 14:41:46.436655 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 27.539569ms to process 1 records E0423 14:41:46.436667 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0423 14:41:46.436672 1 gather.go:177] gatherer "clusterconfig" function "machines" took 27.805527ms to process 0 records I0423 14:41:46.436676 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 27.960642ms to process 0 records I0423 14:41:46.436679 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 27.739462ms to process 0 records I0423 14:41:46.436741 1 tasks_processing.go:74] worker 40 stopped. I0423 14:41:46.436750 1 tasks_processing.go:74] worker 27 stopped. I0423 14:41:46.436761 1 tasks_processing.go:74] worker 5 stopped. I0423 14:41:46.436781 1 tasks_processing.go:74] worker 37 stopped. I0423 14:41:46.436809 1 recorder.go:75] Recording config/network with fingerprint=0a164c53ecb96e08f34f599abb4b706618f318c95c071a1585d5cfc5a1f3852e I0423 14:41:46.436819 1 gather.go:177] gatherer "clusterconfig" function "networks" took 28.037965ms to process 1 records I0423 14:41:46.436889 1 tasks_processing.go:74] worker 60 stopped. I0423 14:41:46.437025 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=a10ddc5609b7e21387be99381b7b3e1b39ec79d6e532ec488cec8a8405b57d88 I0423 14:41:46.437036 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 30.001224ms to process 1 records I0423 14:41:46.437677 1 tasks_processing.go:74] worker 32 stopped. I0423 14:41:46.437875 1 recorder.go:75] Recording config/authentication with fingerprint=e22b5a8b597218338a2a7c673e0e64f6e9a8d65df4f469d5e3e2cc4d64cd1e29 I0423 14:41:46.437888 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 31.573584ms to process 1 records I0423 14:41:46.437913 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 32.665726ms to process 0 records I0423 14:41:46.437923 1 tasks_processing.go:74] worker 36 stopped. I0423 14:41:46.443210 1 tasks_processing.go:74] worker 12 stopped. I0423 14:41:46.443410 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=f1b8add1dcd6e68556cfd42933e5756953a76afa48585d0601d4aa90234f5069 I0423 14:41:46.443493 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=91d3cc843911bd12dd3ec2ec01d924a4ed56a582bbdf2a7784d92fc6804156df I0423 14:41:46.443597 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=a71c9829befa769ce8288dfba5bc5578580413fe46515660c7c16abe7b18ba73 I0423 14:41:46.443624 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=7cbfccb3c609be0c0a83b95b269a58d9740a526e0a0400775800cdd96f7bfe82 I0423 14:41:46.443646 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=5bc33b1711aa320e6066cbd5c8a7099c77ca1dcad319597fa045ef1fe77ea527 I0423 14:41:46.443669 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=6be6bdff24422a72820f462551d3d479083b0aec5bc6958ef22e5cb175b345f7 I0423 14:41:46.443691 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=d592abab383b56bf094ffcb91ce804e3be8301f360d402b638414a92f4ad0de0 I0423 14:41:46.443722 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=febf29d5ef1a8ff7083e1d2b1ca76b694ced4b8d87ab9666dfe08a807249ab19 I0423 14:41:46.443748 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=7b087b15db8420a495efbe54c506023dda80255735af8a0f29c41e1b85a935f4 I0423 14:41:46.443781 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=0b45fa9f30886b310f875ad26d6537da36f3295c8f569921cef768a42fa29578 I0423 14:41:46.443804 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=27c3a19b4adbbbec0bb889afdfa48ad5769af9195dabfebe6961d2070b171911 I0423 14:41:46.443825 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=6e654dcb57f121a0eb8a5de16e9ff80e20e2e6f058fabd117a04df2ab59e59aa I0423 14:41:46.443847 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=19cfadb0e8afcdd3ec1e937bdb24b04da1ffb071fe94b0cbc39cc5f9e98ad5ff I0423 14:41:46.443855 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 38.165047ms to process 13 records I0423 14:41:46.443917 1 tasks_processing.go:74] worker 6 stopped. I0423 14:41:46.444076 1 recorder.go:75] Recording config/oauth with fingerprint=0ecc89cd2f4c771d3b8337c61e24521adb3ccf2d1bfa865bf887267e30e7278e I0423 14:41:46.444086 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 35.346141ms to process 1 records I0423 14:41:46.444177 1 tasks_processing.go:74] worker 33 stopped. I0423 14:41:46.444422 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=342ba74712278f108a79e193243e39c96bf0a75c620f76fe7fc739071db5cb99 I0423 14:41:46.444434 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 36.143627ms to process 1 records W0423 14:41:46.446579 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 14:41:46.446835 1 tasks_processing.go:74] worker 26 stopped. I0423 14:41:46.446927 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=5bf4c8b9eb201334f61a325f05de58d1bac57b6b0cb3df49c7286a4d386f25a6 I0423 14:41:46.446946 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=67c34eecc1b77de73b9e3a5cb2c5beb91c597eb114ec042e2891b387f5667cc1 I0423 14:41:46.446953 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 41.759596ms to process 2 records I0423 14:41:46.449199 1 tasks_processing.go:74] worker 42 stopped. I0423 14:41:46.449212 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 41.723601ms to process 0 records I0423 14:41:46.452786 1 tasks_processing.go:74] worker 35 stopped. I0423 14:41:46.453269 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=e4045216dd9e9092a567fdc9cc95af59b1037ac3eeb6b000beed03c8d81734e4 I0423 14:41:46.453472 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=cad8828a510e964668b356507be4fbe9c8f28e370cd1c561427e6c8f33e6e5ce I0423 14:41:46.453482 1 gather.go:177] gatherer "clusterconfig" function "crds" took 47.733293ms to process 2 records I0423 14:41:46.453558 1 tasks_processing.go:74] worker 39 stopped. I0423 14:41:46.453591 1 recorder.go:75] Recording config/ingress with fingerprint=aee41deba192d661b105dcc2635ebb8f5c49e700ed24f392efe9f1bda8d10568 I0423 14:41:46.453600 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 44.663992ms to process 1 records I0423 14:41:46.456711 1 tasks_processing.go:74] worker 44 stopped. I0423 14:41:46.456723 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 49.236321ms to process 0 records I0423 14:41:46.457023 1 tasks_processing.go:74] worker 24 stopped. I0423 14:41:46.457035 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 51.324275ms to process 0 records I0423 14:41:46.460119 1 tasks_processing.go:74] worker 17 stopped. I0423 14:41:46.461032 1 recorder.go:75] Recording config/node/ip-10-0-0-127.ec2.internal with fingerprint=8258c7eea96ee548486051b4786ed1a1592bf81447fa74d3f4c45ed860f0c9e0 I0423 14:41:46.461175 1 recorder.go:75] Recording config/node/ip-10-0-1-163.ec2.internal with fingerprint=2904d622a563ebc92bd20f1d167ca2b8bd42da2fafebecf09e78fb656a6aba51 I0423 14:41:46.461374 1 recorder.go:75] Recording config/node/ip-10-0-2-76.ec2.internal with fingerprint=b5da37507ac2074b738e0b1f97f67d037527a0acfe327a0b6a7396895cb8f6e3 I0423 14:41:46.461413 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 54.452132ms to process 3 records I0423 14:41:46.461518 1 tasks_processing.go:74] worker 59 stopped. I0423 14:41:46.461602 1 recorder.go:75] Recording config/featuregate with fingerprint=b660db11519b377c37a0ac93d81932790fe3261e763a965aa6d3eafe14487d1b I0423 14:41:46.461623 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 51.960905ms to process 1 records I0423 14:41:46.461642 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0423 14:41:46.461652 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 56.031113ms to process 1 records I0423 14:41:46.461661 1 tasks_processing.go:74] worker 30 stopped. I0423 14:41:46.466278 1 tasks_processing.go:74] worker 16 stopped. I0423 14:41:46.466299 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 58.823697ms to process 0 records I0423 14:41:46.466435 1 tasks_processing.go:74] worker 21 stopped. I0423 14:41:46.466552 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=b638f9909f2d83e1e50b834c6a778444e24e9f7702b19b41cc41df845d207dd2 I0423 14:41:46.466596 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=79c008b3aabd1e2664fdccc1ea792529b2ce9a3367599f0a1cd9277be0d605ee I0423 14:41:46.466620 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=0fdf58fcc8b0d0717d63f48fcddad0b3ce080c4322c39d815c796233148925be I0423 14:41:46.466626 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 58.17832ms to process 3 records I0423 14:41:46.466688 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=370c53cc86ffb8e23cd21a64333ad90d04eff544994b6acaf847b98b041b8af9 I0423 14:41:46.466703 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=e6d305794c74cde9d0a003862bae47ef4ba9ad684b6ef1f26f7be3f026220c1c I0423 14:41:46.466715 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=62581704e81115ad7c20a921bbafc0ecfe4255aba94fec7a2158e57801787a32 I0423 14:41:46.466722 1 tasks_processing.go:74] worker 19 stopped. I0423 14:41:46.466727 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=6efec94ddede854c0e2b5fb3df9aafd9261464d33f0a6c441c8c57cf492b64e4 I0423 14:41:46.466741 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=9f19a46067e0cb3831d4e4c5670b515de3d617436f5756e64e30519bbfe79380 I0423 14:41:46.466757 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=0d6fd318b8b858b9622ade10c74973d0b8877379b219203e38071e0a9bbc4646 I0423 14:41:46.466771 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=6fab070b8e4419a33ba73a0766ff143eb212b68b2db9aaf5a167772b13cf3f94 I0423 14:41:46.466786 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=5d2a0cba7e2d1220b6581bddf5963ad25f88344a55c0f0cd24bd0b5f4aa97a4a I0423 14:41:46.466798 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=7e14bedc443025c17f1bacbceaf39d8d8e7b833e50b5d8f4e0cf565865890105 I0423 14:41:46.466812 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=e7959f11cd03b813e9c637bfbff6dc62563c06ba12da8576c97a069880bda14d I0423 14:41:46.466824 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=1aaa146438f91368c94c24a07f6c6e2b36d4d53f40e6d1b27440b3eadf95d9bb I0423 14:41:46.466830 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 58.181045ms to process 11 records I0423 14:41:46.466875 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e635f00029add8286be17f11ab8c6979c6a4eaf7d43fa26ccac3e389c5442ddc I0423 14:41:46.466882 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 58.657108ms to process 1 records I0423 14:41:46.466887 1 tasks_processing.go:74] worker 38 stopped. E0423 14:41:46.469482 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%279f9a9ebc-d93b-420a-8221-7e5ebb7b2f94%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:35443->172.30.0.10:53: read: connection refused I0423 14:41:46.469495 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%279f9a9ebc-d93b-420a-8221-7e5ebb7b2f94%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:35443->172.30.0.10:53: read: connection refused I0423 14:41:46.474044 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0423 14:41:46.474057 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0423 14:41:46.474067 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0423 14:41:46.478708 1 base_controller.go:82] Caches are synced for ConfigController I0423 14:41:46.478746 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0423 14:41:46.480793 1 tasks_processing.go:74] worker 51 stopped. I0423 14:41:46.481059 1 recorder.go:75] Recording config/version with fingerprint=b835b267b8d5a4faeaa72bab3fe3e13e907c2adeb2c29525a74bbd4853ed6b27 I0423 14:41:46.481071 1 recorder.go:75] Recording config/id with fingerprint=b8fa6877b52a69c6e62eed406d401beb94b89a5f0a87a0b368a8326a1e249474 I0423 14:41:46.481077 1 gather.go:177] gatherer "clusterconfig" function "version" took 74.796149ms to process 2 records I0423 14:41:46.483475 1 tasks_processing.go:74] worker 41 stopped. I0423 14:41:46.483540 1 recorder.go:75] Recording config/olm_operators with fingerprint=ea9251847ad24a42e39dd6c697c2d8beddc78ce3eab45c41571c8afcedf1a714 I0423 14:41:46.483551 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 75.011939ms to process 1 records I0423 14:41:46.484114 1 tasks_processing.go:74] worker 10 stopped. I0423 14:41:46.484342 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=833d3433df4f754bbafb903382eec0e56b9c604e3dcbd353e81e65939b05b354 I0423 14:41:46.484427 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=8f438934cd749638c813fa06b1a96a436d7355a63bd3cf5d9fe1e13b5f9fdec3 I0423 14:41:46.484436 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 75.816176ms to process 2 records I0423 14:41:46.486167 1 tasks_processing.go:74] worker 29 stopped. I0423 14:41:46.489019 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt with fingerprint=ff7867a3f5bfd1c98a01be68128ae6715618fb88d228a7528338d59d95ef773d I0423 14:41:46.489111 1 recorder.go:75] Recording config/pod/openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-0-127.ec2.internal with fingerprint=279b641949cfae3850de609c4da00df33d03fcdee7ed1d8140e17f04fcf279c3 I0423 14:41:46.489195 1 recorder.go:75] Recording config/running_containers with fingerprint=5a223379a824648d8a24b5b90c597930779fa9070b892c3b409bd46966db91f2 I0423 14:41:46.489203 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 77.65681ms to process 3 records I0423 14:41:46.490531 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0423 14:41:46.491804 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 14:41:46.492146 1 tasks_processing.go:74] worker 47 stopped. I0423 14:41:46.492159 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 84.464529ms to process 0 records I0423 14:41:46.493213 1 tasks_processing.go:74] worker 46 stopped. I0423 14:41:46.493232 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0423 14:41:46.493240 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 88.186103ms to process 1 records W0423 14:41:46.493971 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:41996->172.30.0.10:53: read: connection refused I0423 14:41:46.493983 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.10:41996->172.30.0.10:53: read: connection refused I0423 14:41:46.494677 1 tasks_processing.go:74] worker 22 stopped. I0423 14:41:46.494698 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 86.174846ms to process 0 records I0423 14:41:46.502704 1 tasks_processing.go:74] worker 34 stopped. E0423 14:41:46.502716 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0423 14:41:46.502722 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2prtjk59bm7qaqh0k5bvl0hqkh3iqck5-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2prtjk59bm7qaqh0k5bvl0hqkh3iqck5-primary-cert-bundle-secret" not found I0423 14:41:46.502772 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=71c7488ad33475885e3f1ead6081dc0bd52251a6871cfffcd53663fa5b895cc1 I0423 14:41:46.502796 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 96.811431ms to process 1 records I0423 14:41:46.510572 1 tasks_processing.go:74] worker 8 stopped. E0423 14:41:46.510584 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0423 14:41:46.510589 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0423 14:41:46.510592 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0423 14:41:46.510601 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0423 14:41:46.510623 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0423 14:41:46.510629 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0423 14:41:46.510633 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0423 14:41:46.510641 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0423 14:41:46.510676 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0423 14:41:46.510684 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0423 14:41:46.510689 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 102.217872ms to process 7 records I0423 14:41:46.515111 1 base_controller.go:82] Caches are synced for LoggingSyncer I0423 14:41:46.515598 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0423 14:41:46.537763 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0423 14:41:46.542475 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found W0423 14:41:47.447301 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 14:41:48.284131 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0423 14:41:48.447287 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 14:41:48.470114 1 gather_cluster_operator_pods_and_events.go:121] Found 45 pods with 93 containers I0423 14:41:48.470124 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 270600 bytes I0423 14:41:48.470180 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:48.696937 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:48.877752 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:48.886443 1 tasks_processing.go:74] worker 52 stopped. I0423 14:41:48.886572 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=881ed0627dde1f0faee23dafd080b0c93bb6973f40111dd79c75e39312989a9a I0423 14:41:48.886627 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0423 14:41:48.886665 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=f9111a4fd190a5b0809ba977a1892c5f548842850bc961518cbdf25fa3ace774 I0423 14:41:48.886681 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0423 14:41:48.886715 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=c44b8c8b51c86a470b2f383cf690ade8414e47015d6cd571c6fbbbbc23073018 I0423 14:41:48.886739 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0423 14:41:48.886772 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=79ffd50f0b6c88b18d964cfa85af2c44bf37f3a3c25c3559259f875ea1751639 I0423 14:41:48.886811 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=7f8213417a05fd20c7feaf8925b7c526fece8d51f81521e98687f2dde03201ba I0423 14:41:48.886844 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=a6410955466fa6e89a1d0daf42aeb95f881aed4676e65958cafc10e906b7a3a4 I0423 14:41:48.886884 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=f51ef95b4fca398dc6fd25a7baec99535826e45577fcc01ee63896ec23f2ce87 I0423 14:41:48.887179 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=a97544073d349ece083510cdcc2ea16331c24e437d6cc8b8114783cc3ae27403 I0423 14:41:48.887213 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=4aae7b227411aa14e708665099024ab6c50813d35bcf41e7960b497459df2564 I0423 14:41:48.887227 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0423 14:41:48.887255 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=ad1742c935f8f5746cd6e62988209ad27f292019bbdf7436de54a96868995c43 I0423 14:41:48.887271 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0423 14:41:48.887294 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=7c6a13169cedacc15176258eed149e8ad71619f198cd5850b5e9d6289547162a I0423 14:41:48.887307 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0423 14:41:48.887335 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=fadd3565674c03c9ef8451f156ce5f2dedf5f60ec112deabf546341cad531f40 I0423 14:41:48.887349 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0423 14:41:48.887376 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=8db7595c5f4fd2bcf29c067ba4e905e3008883b0fc629c288e35aa0f93642ee2 I0423 14:41:48.887483 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=60eef0156f75c0b095d5ed3d3b0d59c14d6f54ccecd72b5c76075579afd4c7e3 I0423 14:41:48.887496 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0423 14:41:48.887508 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0423 14:41:48.887537 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0423 14:41:48.887568 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=6553ca5ba8927869f7ec607c2d793d2fa58ca6b2b3cbc54f429d8d961c2c5934 I0423 14:41:48.887602 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=310eb9d48a1121d6e0c833abd13fdfb2b00f18fbc3c0833bb5b4067686a04f1c I0423 14:41:48.887616 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0423 14:41:48.887640 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=4bfd03815a5fcebcf126686696bf4fc67f6267e61b4ea8cedf6c1c9a7174884a I0423 14:41:48.887656 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0423 14:41:48.887678 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=670bba02e6564f5e47dd91ef43f87ed5fb64f892fa0c15ad63f099f960b03b15 I0423 14:41:48.887711 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=58f9f61d9b0f56ff0a30ccad3c65bcb58da39d8e090905933081d80ae9d3b2d9 I0423 14:41:48.887737 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=2d213aa59ce4dd9cbc41c1775eba27b52cc32aebb90dad3996230d544fd14ada I0423 14:41:48.887761 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=b987ddcf25c7f793e7405dcf304244a2e651cc60ee15fb2ed33447188737521c I0423 14:41:48.887793 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=fa8cd58e5d2f248aa4f26a06b551d28234c41ffc167eb9962bb6db77f445b149 I0423 14:41:48.887808 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0423 14:41:48.887865 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=d85a228b396eea8cb8f7a00578fd387c197eeaba78e996e4819f33d150b74754 I0423 14:41:48.887892 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0423 14:41:48.887919 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0423 14:41:48.887929 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.478097077s to process 38 records I0423 14:41:49.077297 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:49.276957 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). W0423 14:41:49.446711 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 14:41:49.477277 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:49.677352 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:49.877446 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:50.091735 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:50.275775 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). W0423 14:41:50.446744 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0423 14:41:50.476400 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:50.675724 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:50.875759 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:51.075539 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:51.275472 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-monitoring-operator container cluster-monitoring-operator-698978dbbf-4kj4r pod in namespace openshift-monitoring (previous: false). W0423 14:41:51.446609 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0423 14:41:51.446648 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0423 14:41:51.446663 1 tasks_processing.go:74] worker 0 stopped. E0423 14:41:51.446672 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0423 14:41:51.446684 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0423 14:41:51.446695 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.040371754s to process 1 records I0423 14:41:51.477339 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-state-metrics container kube-state-metrics-7c87d874d5-svd9m pod in namespace openshift-monitoring (previous: false). I0423 14:41:51.676108 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container kube-state-metrics-7c87d874d5-svd9m pod in namespace openshift-monitoring (previous: false). I0423 14:41:51.875540 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container kube-state-metrics-7c87d874d5-svd9m pod in namespace openshift-monitoring (previous: false). I0423 14:41:52.075193 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-5774cc85d7-4wqh4 pod in namespace openshift-monitoring (previous: false). I0423 14:41:52.277352 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-5774cc85d7-txgjs pod in namespace openshift-monitoring (previous: false). I0423 14:41:52.475654 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-67788f859f-q2848 pod in namespace openshift-monitoring (previous: false). I0423 14:41:52.676650 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-67788f859f-xtpt5 pod in namespace openshift-monitoring (previous: false). I0423 14:41:52.892890 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-bb77t pod in namespace openshift-monitoring (previous: false). I0423 14:41:53.074394 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 14:41:53.074410 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-bb77t pod in namespace openshift-monitoring (previous: false). I0423 14:41:53.275863 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-bb77t pod in namespace openshift-monitoring (previous: false). I0423 14:41:53.484968 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-jw8nr pod in namespace openshift-monitoring (previous: false). I0423 14:41:53.676604 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 14:41:53.676619 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-jw8nr pod in namespace openshift-monitoring (previous: false). I0423 14:41:53.876860 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-jw8nr pod in namespace openshift-monitoring (previous: false). I0423 14:41:54.075808 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-mbwg8 pod in namespace openshift-monitoring (previous: false). I0423 14:41:54.277279 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 14:41:54.277291 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-mbwg8 pod in namespace openshift-monitoring (previous: false). I0423 14:41:54.477858 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-mbwg8 pod in namespace openshift-monitoring (previous: false). I0423 14:41:54.677549 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container openshift-state-metrics-6cb79d4dbd-zm48w pod in namespace openshift-monitoring (previous: false). I0423 14:41:54.875059 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container openshift-state-metrics-6cb79d4dbd-zm48w pod in namespace openshift-monitoring (previous: false). I0423 14:41:55.075025 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for openshift-state-metrics container openshift-state-metrics-6cb79d4dbd-zm48w pod in namespace openshift-monitoring (previous: false). I0423 14:41:55.275228 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:55.478848 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:55.685609 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:55.878344 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:56.076927 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:56.277369 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:56.477780 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0423 14:41:56.678005 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:56.875727 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:57.077326 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:57.276269 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:57.476867 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:57.676308 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:57.875820 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0423 14:41:58.075959 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator container prometheus-operator-5df5b8bf6d-c5865 pod in namespace openshift-monitoring (previous: false). I0423 14:41:58.276728 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-operator-5df5b8bf6d-c5865 pod in namespace openshift-monitoring (previous: false). I0423 14:41:58.485063 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-68c48fd977-fzlzq pod in namespace openshift-monitoring (previous: false). I0423 14:41:58.677069 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-68c48fd977-xs5pf pod in namespace openshift-monitoring (previous: false). I0423 14:41:58.875254 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-54f9dcf76-7fd6n pod in namespace openshift-monitoring (previous: false). I0423 14:41:59.076962 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-54f9dcf76-7fd6n pod in namespace openshift-monitoring (previous: false). I0423 14:41:59.267009 1 tasks_processing.go:74] worker 31 stopped. I0423 14:41:59.267049 1 recorder.go:75] Recording config/installplans with fingerprint=f17dbfacc3bfddf27ca3b213b39495434cd4c4e9e3dbd69566ffb3845bbcf539 I0423 14:41:59.267061 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.861579554s to process 1 records I0423 14:41:59.275508 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-54f9dcf76-7fd6n pod in namespace openshift-monitoring (previous: false). I0423 14:41:59.475170 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-54f9dcf76-7fd6n pod in namespace openshift-monitoring (previous: false). I0423 14:41:59.675777 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-54f9dcf76-7fd6n pod in namespace openshift-monitoring (previous: false). I0423 14:41:59.875053 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-54f9dcf76-7fd6n pod in namespace openshift-monitoring (previous: false). I0423 14:42:00.014674 1 tasks_processing.go:74] worker 62 stopped. I0423 14:42:00.015112 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=4aff5860ec073ddfdf1bec96303c96ef7753a0d476f8315de0a2062bde3a2fd5 I0423 14:42:00.015132 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.609089406s to process 1 records I0423 14:42:00.074699 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-54f9dcf76-z5l5h pod in namespace openshift-monitoring (previous: false). I0423 14:42:00.276983 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-54f9dcf76-z5l5h pod in namespace openshift-monitoring (previous: false). I0423 14:42:00.476378 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-54f9dcf76-z5l5h pod in namespace openshift-monitoring (previous: false). I0423 14:42:00.676402 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-54f9dcf76-z5l5h pod in namespace openshift-monitoring (previous: false). I0423 14:42:00.876127 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-54f9dcf76-z5l5h pod in namespace openshift-monitoring (previous: false). I0423 14:42:01.077298 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-54f9dcf76-z5l5h pod in namespace openshift-monitoring (previous: false). I0423 14:42:01.276452 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-6r5cn pod in namespace openshift-network-console (previous: false). I0423 14:42:01.476217 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-xw2s5 pod in namespace openshift-network-console (previous: false). I0423 14:42:01.676180 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-lzbtt pod in namespace openshift-console-operator (previous: true). I0423 14:42:01.875601 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-lzbtt pod in namespace openshift-console-operator (previous: false). I0423 14:42:02.079232 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-58b747998f-cljvt pod in namespace openshift-console (previous: false). I0423 14:42:02.277618 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-679f8b9658-8vjd6 pod in namespace openshift-console (previous: false). I0423 14:42:02.476733 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-679f8b9658-c5spj pod in namespace openshift-console (previous: false). I0423 14:42:02.678697 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-9pmb8 pod in namespace openshift-console (previous: false). I0423 14:42:02.876951 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-djhwk pod in namespace openshift-console (previous: false). I0423 14:42:03.080603 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-929w7 pod in namespace openshift-dns (previous: false). I0423 14:42:03.277770 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-929w7 pod in namespace openshift-dns (previous: false). I0423 14:42:03.478061 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-cps8z pod in namespace openshift-dns (previous: false). I0423 14:42:03.677026 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-cps8z pod in namespace openshift-dns (previous: false). I0423 14:42:03.877260 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-wxthm pod in namespace openshift-dns (previous: false). I0423 14:42:04.075823 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-wxthm pod in namespace openshift-dns (previous: false). I0423 14:42:04.275936 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-bm4kh pod in namespace openshift-dns (previous: false). I0423 14:42:04.477747 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 14:42:04.477762 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-g45dr pod in namespace openshift-dns (previous: false). I0423 14:42:04.676109 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 14:42:04.676122 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-r7td7 pod in namespace openshift-dns (previous: false). I0423 14:42:04.875686 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0423 14:42:04.875699 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-599475d84c-2lpjx pod in namespace openshift-image-registry (previous: false). I0423 14:42:05.076622 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-599475d84c-x6kvs pod in namespace openshift-image-registry (previous: false). I0423 14:42:05.277637 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6cdc54b67b-t5j5t pod in namespace openshift-image-registry (previous: true). I0423 14:42:05.497259 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6cdc54b67b-t5j5t pod in namespace openshift-image-registry (previous: false). I0423 14:42:05.677792 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-bkv4m pod in namespace openshift-image-registry (previous: false). I0423 14:42:05.877755 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-nhstg pod in namespace openshift-image-registry (previous: false). I0423 14:42:06.075983 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-vlk9n pod in namespace openshift-image-registry (previous: false). I0423 14:42:06.275641 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5c464d6b7f-76x9c pod in namespace openshift-ingress (previous: false). I0423 14:42:06.476354 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5c464d6b7f-vqh9q pod in namespace openshift-ingress (previous: false). I0423 14:42:06.678519 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6458c485df-44t2t pod in namespace openshift-ingress (previous: false). I0423 14:42:06.870546 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6458c485df-44t2t pod in namespace openshift-ingress for failing operator router (previous: false): "pods \"router-default-6458c485df-44t2t\" not found" I0423 14:42:06.870562 1 gather_cluster_operator_pods_and_events.go:280] Error: "pods \"router-default-6458c485df-44t2t\" not found" I0423 14:42:06.870588 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-4wf5j pod in namespace openshift-ingress-canary (previous: false). I0423 14:42:07.078936 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-q28j4 pod in namespace openshift-ingress-canary (previous: false). I0423 14:42:07.276013 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-xf9xk pod in namespace openshift-ingress-canary (previous: false). I0423 14:42:07.475918 1 tasks_processing.go:74] worker 58 stopped. I0423 14:42:07.476588 1 recorder.go:75] Recording events/openshift-monitoring with fingerprint=bfa54ab8d1e742dd5793c31b76d353321eb430a213cab9296a3b98219f629281 I0423 14:42:07.476634 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=7932595c0cf1a0bda5701ceeb0822d6a07f7ac3078a69a56a2b75bb366e8f268 I0423 14:42:07.476725 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=29cfb6258638f7218df8c4c90c30f6612c40fc926b6a99905932a5ea31773212 I0423 14:42:07.476806 1 recorder.go:75] Recording events/openshift-console with fingerprint=03909ea19c410c0ec546dc10fc9be093bec26e444ab6c5df57172d4fdc91f18b I0423 14:42:07.476830 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=e73266fb36ae20b810dc10a4f18b4db52139e1685e0f2a5a67554c5cd70a4089 I0423 14:42:07.476873 1 recorder.go:75] Recording events/openshift-dns with fingerprint=1b5c9498c4d42488637b5b858822b2f21402912acbed5dc277e90800277757f3 I0423 14:42:07.477033 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=6b201453efffa933abc11b8b091a7a7f8544afccec0a2c002472084545c22a82 I0423 14:42:07.477059 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=4f9d47f79e75b7e917aff42763a604783d65b3c4df17a185e9edd5fef29c97ff I0423 14:42:07.477151 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=59fa282fb89b7345944774d2db4ae069f7aa06018e214cdaaae67b9f71610fe7 I0423 14:42:07.477167 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=7e2de850adddad0d5d4a9854f057cf72a0e8a89a6e857b7c2137021efbbf1809 I0423 14:42:07.477176 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/init-config-reloader_current.log with fingerprint=06a43e67b635f169e3ecb266e819feea6dcb9205dd5d24effea642a2bf0dbd57 I0423 14:42:07.477199 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/alertmanager_current.log with fingerprint=eb7e5027d44ebd03013cd4f4b4238aeaf94907b93a831ce8bf86059ab111245d I0423 14:42:07.477213 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/config-reloader_current.log with fingerprint=68df0bbf05e8cb33a6889dbbedd99d6e794f8030377ec592d0555d0ec0ceaf76 I0423 14:42:07.477223 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-web_current.log with fingerprint=95197d5ff9e95a3ed4dc7b7317c5bbaf687787064851d512c9e27f9578aa76c8 I0423 14:42:07.477230 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy_current.log with fingerprint=71b81b4a3b6b837859039439fe7349b73b681befc924dcf0cf597fb83f4d9f85 I0423 14:42:07.477236 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-metric_current.log with fingerprint=dfaa7ed4c918edd4e391c1a8d63b8c5b41239fb3bf2713301b5dd9ecbaacc145 I0423 14:42:07.477240 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/prom-label-proxy_current.log with fingerprint=e66ed847edf7e6c5ad664a38cc76e208cd31b3f9e9b36e688a1b66ba35c7d814 I0423 14:42:07.477247 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/init-config-reloader_current.log with fingerprint=e87fe66b1bc7eef1a0fa496ea27aface1790fa476862cb29404da9041636bbcc I0423 14:42:07.477269 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/alertmanager_current.log with fingerprint=abba240ee52ed9b1cfc601c4426d61e40c2fd1c54aba91fda97c597a41b112fe I0423 14:42:07.477283 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/config-reloader_current.log with fingerprint=984a6cb862da11cb454ff6d6d7bba6697baee23db8ede116a4cbd7d40091019a I0423 14:42:07.477292 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-web_current.log with fingerprint=a92a02f090e6b58c6fdfbc8c2afce0fe46b747ebd9825f84527e08edbc4c7e12 I0423 14:42:07.477297 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy_current.log with fingerprint=b08794758ace428c0e0caac690d74b45e6872927a437eaf3b3f1aab77b3cacab I0423 14:42:07.477303 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-metric_current.log with fingerprint=2f8d4c922683bc01c66e5e13ebaffd0cf54b5a71c30ad641b55f99dfafeb63a8 I0423 14:42:07.477307 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/prom-label-proxy_current.log with fingerprint=f2d957269ae8c8869a255b8e5f46bf2dd979a1a3d38936434c394112d93c969d I0423 14:42:07.477421 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/cluster-monitoring-operator-698978dbbf-4kj4r/cluster-monitoring-operator_current.log with fingerprint=345a5d68a0639e2ce700290ccc2da9c8dc21ff4387120ac8dcd3a050ea6c1126 I0423 14:42:07.477440 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-svd9m/kube-state-metrics_current.log with fingerprint=a3e031e644c001edf24bb4780712f1ec6aaa248d156142c25f7198343fa96074 I0423 14:42:07.477449 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-svd9m/kube-rbac-proxy-main_current.log with fingerprint=57d597db7b967c95e5d591f7d13c4f4294db6e0dba1080209fe932b6dc1634be I0423 14:42:07.477455 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-7c87d874d5-svd9m/kube-rbac-proxy-self_current.log with fingerprint=a23e10133d49566c2b77c7fd1029de6575d682640260019591773308f3f65a65 I0423 14:42:07.477464 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-5774cc85d7-4wqh4/metrics-server_current.log with fingerprint=f987c0b4b3c3bd13d0f512e57ba40b64d6cb31f4eb80fc53911860b592f011aa I0423 14:42:07.477470 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-5774cc85d7-txgjs/metrics-server_current.log with fingerprint=d40216e184ea2f229e36804e5163b20237e645f436ada7f9794ba9c6b05d8966 I0423 14:42:07.477480 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-67788f859f-q2848/monitoring-plugin_current.log with fingerprint=ef78176b850dc5a0d795e36e3f4f82c3f855f0f398e583b86e19a5adc4abd072 I0423 14:42:07.477491 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-67788f859f-xtpt5/monitoring-plugin_current.log with fingerprint=191671511f6e026ae8e3856d005718cfe0f279fa53058119ce59c052a3989174 I0423 14:42:07.477516 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-bb77t/node-exporter_current.log with fingerprint=6c86cbc34e4daa8bd5cc451575bcb0ab177127eb5450d1fd34358cca3e57dc45 I0423 14:42:07.477525 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-bb77t/kube-rbac-proxy_current.log with fingerprint=bd2d5e3b86bb294d6139dc862897e263705631f26c4a865a84af97193a4966ad I0423 14:42:07.477560 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-jw8nr/node-exporter_current.log with fingerprint=7f548880b0ebab3fb84c8042c4ca85ded86395d74f198a23c0187414a394ef9f I0423 14:42:07.477590 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-jw8nr/kube-rbac-proxy_current.log with fingerprint=9c516ac541773e4f1ce984f2704c88540b16d7d94f82df86b6492f0277ab7a96 I0423 14:42:07.477618 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-mbwg8/node-exporter_current.log with fingerprint=7963af0e6b762ef728b23bb272805bd937129cf2f76a9b8266b5190f34b5bfe2 I0423 14:42:07.477627 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-mbwg8/kube-rbac-proxy_current.log with fingerprint=5f76758158415ab2432e44f4c22cbc9cbe596612ee98440253c362113f24d3f2 I0423 14:42:07.477635 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-zm48w/kube-rbac-proxy-main_current.log with fingerprint=f60e5b91969cef14a226dfce9043f04436f40747f663fa213ac857aae06f2a10 I0423 14:42:07.477640 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-zm48w/kube-rbac-proxy-self_current.log with fingerprint=3446901d81c44d652f488194db087c4cc7c94e606dcece8e078bcf81aa5d7714 I0423 14:42:07.477660 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-6cb79d4dbd-zm48w/openshift-state-metrics_current.log with fingerprint=4dec596da5841737a43104853aa3b0cf925be275c9dcac4afaa45dab41f537de I0423 14:42:07.477669 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/init-config-reloader_current.log with fingerprint=41a1326a0bbe1df2d5cc508e73b0c0ec2202831b4a7764f3525415bf0a10af4a I0423 14:42:07.477768 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/prometheus_current.log with fingerprint=9911b8a292de10107ce79539a42178572e1493e12f66952a56f29cf2b52d28ed I0423 14:42:07.477783 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/config-reloader_current.log with fingerprint=5d9a46e240e1e41bb0334d171698124cfcba2cb3ef3e9c9ee9cb655f7d8d21e1 I0423 14:42:07.477794 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/thanos-sidecar_current.log with fingerprint=9b53c41bbf8edaf56c9c67538cd438dfb4ba04bd22728554115bd15ac441f09a I0423 14:42:07.477799 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-web_current.log with fingerprint=0f871fbd14ccebb679339c1c020dc5c5ff7dd354d17994cc5ed3d1182fe1282b I0423 14:42:07.477805 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy_current.log with fingerprint=df5d1372241333805184ae43a4e2af46ef17b591307a0e9f654612600bb9096c I0423 14:42:07.477811 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-thanos_current.log with fingerprint=0a3eaa461c78dab7db4266e6a1228a044e2e2d37b41e5b3c4c92c5bd199ea4a9 I0423 14:42:07.477817 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/init-config-reloader_current.log with fingerprint=882a585f4269b11ce70ea7be43d29ba433cb38f1fbe77de7887eecf3baea4915 I0423 14:42:07.477923 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/prometheus_current.log with fingerprint=cc08ee77bf98de5cfd3aae871aec8b7005abe78bc21f667d8d5fdbbfd1f6667e I0423 14:42:07.477936 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/config-reloader_current.log with fingerprint=1a82ec07aac8781642a71c20a07d9644810da31eb51a0ce26ad34e8e63ad1dd5 I0423 14:42:07.477947 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/thanos-sidecar_current.log with fingerprint=da9317b27d35de61fb03b3ef4e148dbe210d9df46283329ec195e8a2b018cc55 I0423 14:42:07.477952 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-web_current.log with fingerprint=4341cc231940ef86118121b6f8f9b9be6e10a9e881440784146a3b25dc4144d6 I0423 14:42:07.477959 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy_current.log with fingerprint=71df63d59a6fbfd1311424dee25eb9a53e1c9d7386988bf27ea8726642896915 I0423 14:42:07.477965 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-thanos_current.log with fingerprint=8e1b0b5fd3f2084c222e23f27dafe50a770e8621e956b6ec0f2a2f51642a37bc I0423 14:42:07.478040 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-5df5b8bf6d-c5865/prometheus-operator_current.log with fingerprint=fb7baee33d76fa2b20578f34944334f52f0dc57d93badc5c6f4519f19a049eaf I0423 14:42:07.478048 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-5df5b8bf6d-c5865/kube-rbac-proxy_current.log with fingerprint=b530fd022aa79f9bb3e1e32d8847f52ef991182d880ebfd260c4e544b923b497 I0423 14:42:07.478054 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-68c48fd977-fzlzq/prometheus-operator-admission-webhook_current.log with fingerprint=44f5649f9cc56ae8c98cb5d1a0e7d570d112b0949f2417a57dd727674cf44c79 I0423 14:42:07.478062 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-68c48fd977-xs5pf/prometheus-operator-admission-webhook_current.log with fingerprint=5d40f1e58197e99f6105f1c15d0cbc29090d978bdaa32df0bdef8e714a96dcae I0423 14:42:07.478092 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-7fd6n/thanos-query_current.log with fingerprint=f1b75784e68b6e36e6b1a6cc340641e0ee040656a9cdda3184dff6bfd4b4d351 I0423 14:42:07.478111 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-7fd6n/kube-rbac-proxy-web_current.log with fingerprint=103d1bb0659d79fad572db76f550664e9df023264959a312cf71ea1b794e36aa I0423 14:42:07.478117 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-7fd6n/kube-rbac-proxy_current.log with fingerprint=f0bad908171e1d9274e5bbd255e882897f95853a0a796d38deda6e5f3b662eaa I0423 14:42:07.478121 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-7fd6n/prom-label-proxy_current.log with fingerprint=6e30094fd3f0670d305358f17b9e697edfcac774d9896e58821edb6565cf10d7 I0423 14:42:07.478127 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-7fd6n/kube-rbac-proxy-rules_current.log with fingerprint=2cf4d5ec02abd825b671205134242471b690e9012839b5a326f92d2b6d58693b I0423 14:42:07.478133 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-7fd6n/kube-rbac-proxy-metrics_current.log with fingerprint=db1b03df24e6104cfc1d27992915730a165c4367ada4cabab3eddd01d00de3d3 I0423 14:42:07.478164 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-z5l5h/thanos-query_current.log with fingerprint=1947884eda7a2b98e0467c6f550f92a7a9c76eacfdb7094c028eb7e6af970925 I0423 14:42:07.478179 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-z5l5h/kube-rbac-proxy-web_current.log with fingerprint=3a899db1f15be5b8ffa9caa80233e14e2a71dd7ea9873841f7a0d730b9f32e85 I0423 14:42:07.478185 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-z5l5h/kube-rbac-proxy_current.log with fingerprint=9a0e431f6df27c6739b0d9657c6c529894ef9d0628fb6b51d33d4cf990d3ef61 I0423 14:42:07.478189 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-z5l5h/prom-label-proxy_current.log with fingerprint=e24d4ee17718e44f3557ab0cd05c0c10d7aea259cb4b0e6674c7e877d497eebb I0423 14:42:07.478194 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-z5l5h/kube-rbac-proxy-rules_current.log with fingerprint=caf12cc20e9140623c23eae7b4ead7e72ab1337a7fc3a200639e3a2a0a35cfe8 I0423 14:42:07.478200 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-54f9dcf76-z5l5h/kube-rbac-proxy-metrics_current.log with fingerprint=015baf30af6ecba4fc8c09fa5c59fc24dd54a3f5a4955d519e528a3525b5acdb I0423 14:42:07.478206 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-6r5cn/networking-console-plugin_current.log with fingerprint=e8f1b0686067cde508851687ed7dde47462ef2d3492faa03795f432b23209a27 I0423 14:42:07.478211 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-xw2s5/networking-console-plugin_current.log with fingerprint=9689d2d4699679f9e12d82a0aa0c3c0a4e621c3d27b3ca3cc200ea0139e5ad98 I0423 14:42:07.478346 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt with fingerprint=ff7867a3f5bfd1c98a01be68128ae6715618fb88d228a7528338d59d95ef773d E0423 14:42:07.478364 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json" was already recorded and had the fingerprint "ff7867a3f5bfd1c98a01be68128ae6715618fb88d228a7528338d59d95ef773d", overwriting with the record having fingerprint "ff7867a3f5bfd1c98a01be68128ae6715618fb88d228a7528338d59d95ef773d" W0423 14:42:07.478377 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json" because of the warning: warning: the record with the same fingerprint "ff7867a3f5bfd1c98a01be68128ae6715618fb88d228a7528338d59d95ef773d" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json" I0423 14:42:07.478400 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-lzbtt/console-operator_previous.log with fingerprint=660e95fa5a62c7059756a18007a6690235cf8b8d704fbbb9b4b31268b0ac74e3 I0423 14:42:07.478571 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-lzbtt/console-operator_current.log with fingerprint=283fb34f57107c1138a3526f8737ed595662d861c01f35c3937f396f9f4fba07 I0423 14:42:07.478584 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-58b747998f-cljvt/console_current.log with fingerprint=0a590c382da26ffc9b8ff126e3ea3284651a16a809fe7e4ee7ded6dcbcee7d68 I0423 14:42:07.478593 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-679f8b9658-8vjd6/console_current.log with fingerprint=2c9c90fc93a19c948d31daf782250e602da6b1a769743e29eaa78fbae1e62014 I0423 14:42:07.478603 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-679f8b9658-c5spj/console_current.log with fingerprint=41bc4941b8d1992046d62bbd2eae674cdcb8ae46d90a0e62dabc6b3e642112d2 I0423 14:42:07.478619 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-9pmb8/download-server_current.log with fingerprint=b87f9a1796572f03e3a2cf0dcb75181a0425542c4108ade0375695159a0f4d38 I0423 14:42:07.478625 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-djhwk/download-server_current.log with fingerprint=651e640deb5b7f299aa4719bf2b20f1572972273e4f924d98a5365d6e28616fe I0423 14:42:07.478724 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-929w7 with fingerprint=f53b91f54a91c716cb6e76f7e10b6e3bd0627ee23d4b2fcce72a1ceba242b8e5 I0423 14:42:07.478732 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-929w7/dns_current.log with fingerprint=a0ec9a8a05cdaf6b9dd4849a4b20f09233a58123d83b0e2b6c3853e6d198f2d2 I0423 14:42:07.478739 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-929w7/kube-rbac-proxy_current.log with fingerprint=5ad8f66011cfefc5f455276b1c609f47385e72c0a86e842c579782587803d3a5 I0423 14:42:07.478809 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-cps8z with fingerprint=de4180644d1316b44879c50dc64b00b6834e9c483e463618bbf418609eacee7f I0423 14:42:07.478816 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-cps8z/dns_current.log with fingerprint=dfdb22a7a355c245fe57867ef488c65e4ff3f33d2b6f0baa4fb76a14aa3615f2 I0423 14:42:07.478823 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-cps8z/kube-rbac-proxy_current.log with fingerprint=3b57d7f7f6b83d035909ee0eb1eaf2bf98da9b94f7f1b9e11dbadc5fcb14ed94 I0423 14:42:07.478892 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-wxthm with fingerprint=031f69983aecbde131ef0f1cc5abab09ed18bb1029992b4d8390299795662636 I0423 14:42:07.478911 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-wxthm/dns_current.log with fingerprint=b662ec52251bd039457b020eccf3d0b6afd88ed94873c7ffc6a452d90a4d3947 I0423 14:42:07.478919 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-wxthm/kube-rbac-proxy_current.log with fingerprint=7478721d0e9a35a5cacaa5003540c2a6975b704de20fd86132f2d69216a03208 I0423 14:42:07.478957 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-599475d84c-2lpjx/registry_current.log with fingerprint=da85126aeeeaa318ed3a9a53344f3f538c2d4e8bcd2bf34fdf171bb4f2d8cd79 I0423 14:42:07.478979 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-599475d84c-x6kvs/registry_current.log with fingerprint=ccdfe5a3ede93bdf089ffc3c01684fc39fb3daeeb661432c6046770c26629741 I0423 14:42:07.479087 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-6cdc54b67b-t5j5t with fingerprint=d5fea53af1ac67c75d5ebd3aa8b5cf6a1ddeb83dad6471b964bf0e52ef26675c I0423 14:42:07.479123 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-6cdc54b67b-t5j5t/registry_previous.log with fingerprint=e0e0cc997e52c2376940ec10bc3057c1be122c4b5f6eb8daa375f8f399ea1e60 I0423 14:42:07.479153 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-6cdc54b67b-t5j5t/registry_current.log with fingerprint=d39f57ccc9692ea26bd06207e4ad749e7af9b1d8b41a255c61e993d666d2f428 I0423 14:42:07.479160 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-bkv4m/node-ca_current.log with fingerprint=df1dc28fcde956070fd8f5739282496b7e7d6668f973b139783033ca1008a144 I0423 14:42:07.479164 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-nhstg/node-ca_current.log with fingerprint=db9fd889d7d5f3911e560d87b87e9507140325d6306cf1696cdbbe478f4764c2 I0423 14:42:07.479169 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-vlk9n/node-ca_current.log with fingerprint=3dcc4b73add41e37e0850550e0677721e052de1fa59447551d74225717f43a34 I0423 14:42:07.479207 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5c464d6b7f-76x9c/router_current.log with fingerprint=5861f488587a4834031b3b0ffd76640e7706bac128b1293e391b5e24e01d04e9 I0423 14:42:07.479249 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5c464d6b7f-vqh9q/router_current.log with fingerprint=6605d5acde2628525a52220796049e7e3869cde05c508114fd54ad189a503014 I0423 14:42:07.479303 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-4wf5j with fingerprint=ea0fe75ea8fafe6a054431fd055d15c2cc9a438d755d073b40c3fc305a6d0076 I0423 14:42:07.479310 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-4wf5j/serve-healthcheck-canary_current.log with fingerprint=f9dfc9534599b7c5256c3009af95cb1ea93f8d99f9a96b1d5f0a2d47aea0c1e0 I0423 14:42:07.479359 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-q28j4 with fingerprint=daf88308a2a1e6a4a53b7bdf404ef4262a4aa62f16dc28f912d4d965dd5511fb I0423 14:42:07.479365 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-q28j4/serve-healthcheck-canary_current.log with fingerprint=dd06c57b5e85cbd4f0e31813e9335cad44b70087a0409c2be3d941c206fe7241 I0423 14:42:07.479416 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-xf9xk with fingerprint=fb37d765a2563f7264388b9bde11557e7c8ff6b847ea4c0769e5bde6cc28e565 I0423 14:42:07.479422 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-xf9xk/serve-healthcheck-canary_current.log with fingerprint=33ec817a1e9fc756832b85eb61dfe9e9bd11292b7f67bf6a8e26299098570f87 I0423 14:42:07.479427 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 21.067729963s to process 106 records E0423 14:42:07.479487 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 21.074s with: function \"silenced_alerts\" failed with an error, function \"active_alerts\" failed with an error, function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"machines\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json\"" I0423 14:42:07.480591 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "silenced_alerts" failed with an error, function "active_alerts" failed with an error, function "tsdb_status" failed with an error, function "metrics" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "machines" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-lzbtt.json" I0423 14:42:07.480604 1 periodic.go:209] Running workloads gatherer I0423 14:42:07.480619 1 tasks_processing.go:45] number of workers: 2 I0423 14:42:07.480625 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 14:42:07.480628 1 tasks_processing.go:71] worker 1 working on helmchart_info task. I0423 14:42:07.480639 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 14:42:07.480655 1 tasks_processing.go:71] worker 0 working on workload_info task. I0423 14:42:07.486961 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-127.ec2.internal... I0423 14:42:07.486968 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-76.ec2.internal... I0423 14:42:07.486985 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-163.ec2.internal... I0423 14:42:07.506487 1 tasks_processing.go:74] worker 1 stopped. I0423 14:42:07.506507 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 25.84727ms to process 0 records I0423 14:42:12.644269 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0423 14:42:12.671078 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0423 14:42:12.679436 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0423 14:42:12.688128 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0423 14:42:12.696420 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0423 14:42:12.705227 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (9ms) I0423 14:42:12.713763 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (9ms) I0423 14:42:12.722456 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (9ms) I0423 14:42:12.730970 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (9ms) I0423 14:42:12.739274 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (8ms) I0423 14:42:12.747499 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (8ms) I0423 14:42:12.755300 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (8ms) I0423 14:42:12.778772 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (23ms) I0423 14:42:12.881267 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (102ms) I0423 14:42:12.979866 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (99ms) I0423 14:42:13.080885 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (101ms) I0423 14:42:13.180672 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0423 14:42:13.280139 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (99ms) I0423 14:42:13.379669 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (100ms) I0423 14:42:13.479304 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0423 14:42:13.580103 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0423 14:42:13.679851 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (100ms) I0423 14:42:13.781711 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (102ms) I0423 14:42:13.879090 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (97ms) I0423 14:42:13.979808 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0423 14:42:14.079684 1 gather_workloads_info.go:387] No image sha256:ce98d5d844bfc2ba8de1893866ad38166c95157d54abd8192b181e819bc50bb5 (100ms) I0423 14:42:14.180587 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (101ms) I0423 14:42:14.280023 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (99ms) I0423 14:42:14.379754 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (100ms) I0423 14:42:14.479806 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0423 14:42:14.579490 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (100ms) I0423 14:42:14.680599 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (101ms) I0423 14:42:14.780512 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (100ms) I0423 14:42:14.880321 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0423 14:42:14.979477 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (99ms) I0423 14:42:15.080061 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (101ms) I0423 14:42:15.180692 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (101ms) I0423 14:42:15.282537 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (102ms) I0423 14:42:15.400982 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (118ms) I0423 14:42:15.502029 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (101ms) I0423 14:42:15.579620 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (78ms) I0423 14:42:15.682331 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (103ms) I0423 14:42:15.786138 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (104ms) I0423 14:42:15.879585 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (93ms) I0423 14:42:15.979959 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0423 14:42:16.079448 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0423 14:42:16.180206 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (101ms) I0423 14:42:16.280589 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0423 14:42:16.379419 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (99ms) I0423 14:42:16.480724 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (101ms) I0423 14:42:16.480761 1 tasks_processing.go:74] worker 0 stopped. I0423 14:42:16.481050 1 recorder.go:75] Recording config/workload_info with fingerprint=aed3d0c1f8e26cbf8e8fe292a79ef1c27c7b74fbc95cbe89aab36daddf7ce581 I0423 14:42:16.481069 1 gather.go:177] gatherer "workloads" function "workload_info" took 9.000097458s to process 1 records I0423 14:42:16.481089 1 periodic.go:256] Periodic gather workloads completed in 9s I0423 14:42:16.481100 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0423 14:42:16.481110 1 periodic.go:209] Running conditional gatherer I0423 14:42:16.487095 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0423 14:42:16.699841 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0423 14:42:17.744306 1 conditional_gatherer.go:374] alert "TargetDown" has state "pending" I0423 14:42:17.744322 1 conditional_gatherer.go:374] alert "TargetDown" has state "pending" I0423 14:42:17.744326 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744329 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744331 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744333 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744336 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744338 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744340 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744342 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744344 1 conditional_gatherer.go:374] alert "KubePodNotReady" has state "pending" I0423 14:42:17.744346 1 conditional_gatherer.go:374] alert "KubeStatefulSetReplicasMismatch" has state "pending" I0423 14:42:17.744348 1 conditional_gatherer.go:374] alert "KubeDaemonSetRolloutStuck" has state "pending" I0423 14:42:17.744350 1 conditional_gatherer.go:374] alert "KubeDaemonSetRolloutStuck" has state "pending" I0423 14:42:17.744352 1 conditional_gatherer.go:374] alert "KubeDaemonSetRolloutStuck" has state "pending" I0423 14:42:17.744354 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744356 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744358 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744360 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744362 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744364 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744366 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744368 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744370 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744372 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744374 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744376 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744378 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744380 1 conditional_gatherer.go:374] alert "KubeContainerWaiting" has state "pending" I0423 14:42:17.744382 1 conditional_gatherer.go:374] alert "KubePdbNotEnoughHealthyPods" has state "pending" I0423 14:42:17.744384 1 conditional_gatherer.go:374] alert "KubePdbNotEnoughHealthyPods" has state "pending" I0423 14:42:17.744386 1 conditional_gatherer.go:374] alert "KubePdbNotEnoughHealthyPods" has state "pending" I0423 14:42:17.744388 1 conditional_gatherer.go:374] alert "KubePdbNotEnoughHealthyPods" has state "pending" I0423 14:42:17.744391 1 conditional_gatherer.go:374] alert "PrometheusKubernetesListWatchFailures" has state "pending" I0423 14:42:17.744394 1 conditional_gatherer.go:374] alert "PrometheusKubernetesListWatchFailures" has state "pending" I0423 14:42:17.744396 1 conditional_gatherer.go:374] alert "ClusterMonitoringOperatorReconciliationErrors" has state "pending" I0423 14:42:17.744398 1 conditional_gatherer.go:374] alert "KubePodNotScheduled" has state "pending" I0423 14:42:17.744400 1 conditional_gatherer.go:374] alert "Watchdog" has state "firing" I0423 14:42:17.744474 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0423 14:42:17.750580 1 conditional_gatherer.go:392] cluster version is '4.20.8' I0423 14:42:17.750608 1 tasks_processing.go:45] number of workers: 3 I0423 14:42:17.750621 1 tasks_processing.go:69] worker 2 listening for tasks. I0423 14:42:17.750625 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0423 14:42:17.750630 1 tasks_processing.go:69] worker 0 listening for tasks. I0423 14:42:17.750636 1 tasks_processing.go:69] worker 1 listening for tasks. I0423 14:42:17.750642 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0423 14:42:17.750644 1 tasks_processing.go:74] worker 1 stopped. I0423 14:42:17.750661 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0423 14:42:17.750707 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0423 14:42:17.750720 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 759ns to process 1 records I0423 14:42:17.750753 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0423 14:42:17.750760 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.489µs to process 1 records I0423 14:42:17.750765 1 tasks_processing.go:74] worker 0 stopped. I0423 14:42:17.750937 1 tasks_processing.go:74] worker 2 stopped. I0423 14:42:17.750951 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 259.684µs to process 0 records I0423 14:42:17.750975 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0423 14:42:17.750999 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0423 14:42:17.779321 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=383aef4c0fcf680d0756b0a0afa913f8e4021be6c7f40fdd7306c759f276c5dc I0423 14:42:17.779488 1 diskrecorder.go:70] Writing 214 records to /var/lib/insights-operator/insights-2026-04-23-144217.tar.gz I0423 14:42:17.794074 1 diskrecorder.go:51] Wrote 214 records to disk in 14ms I0423 14:42:17.794101 1 periodic.go:278] Gathering cluster info every 2h0m0s I0423 14:42:17.794116 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0423 14:43:16.405481 1 diskrecorder.go:223] Found files to send: insights-2026-04-23-144217.tar.gz I0423 14:43:16.405509 1 insightsuploader.go:150] Checking archives to upload periodically every 16m26.827456567s I0423 14:43:16.405520 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0423 14:43:16.421012 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0423 14:43:16.618427 1 requests.go:87] Successfully reported id=2026-04-23T14:43:16Z x-rh-insights-request-id=7418df36fd844caba3c4384b75ffa840, wrote=111219 I0423 14:43:16.618471 1 insightsuploader.go:187] Uploaded report successfully in 212.946751ms I0423 14:43:16.618488 1 controller.go:128] Initializing last reported time to 2026-04-23T14:43:16Z I0423 14:43:16.618503 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0423 14:43:16.618511 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0423 14:43:16.618518 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0423 14:43:16.624251 1 controller.go:512] The operator is healthy I0423 14:43:46.423800 1 controller.go:512] The operator is healthy I0423 14:44:19.856848 1 insightsreport.go:137] Pulling report from smart-proxy I0423 14:44:19.856869 1 insightsreport.go:149] Retrieving report I0423 14:44:19.863591 1 requests.go:110] Retrieving report for cluster: 9f9a9ebc-d93b-420a-8221-7e5ebb7b2f94 I0423 14:44:19.863601 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/9f9a9ebc-d93b-420a-8221-7e5ebb7b2f94/reports I0423 14:44:19.866248 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/9f9a9ebc-d93b-420a-8221-7e5ebb7b2f94/reports I0423 14:44:20.027289 1 insightsreport.go:184] Report retrieved I0423 14:44:20.054663 1 insightsreport.go:239] Report retrieved correctly I0423 14:45:46.425532 1 controller.go:512] The operator is healthy I0423 14:46:46.398020 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0423 14:46:46.408267 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 14:46:46.408289 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0423 14:46:46.413938 1 secretconfigobserver.go:119] support secret does not exist I0423 14:47:46.427195 1 controller.go:512] The operator is healthy I0423 14:49:46.421791 1 controller.go:512] The operator is healthy I0423 14:51:46.395720 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 14:51:46.414812 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0423 14:51:46.420246 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 14:51:46.420264 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0423 14:51:46.421033 1 controller.go:512] The operator is healthy I0423 14:51:46.424626 1 secretconfigobserver.go:119] support secret does not exist I0423 14:53:46.422234 1 controller.go:512] The operator is healthy I0423 14:55:46.425427 1 controller.go:512] The operator is healthy I0423 14:56:46.425150 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0423 14:56:46.431828 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 14:56:46.431850 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0423 14:56:46.436503 1 secretconfigobserver.go:119] support secret does not exist I0423 14:57:46.425146 1 controller.go:512] The operator is healthy I0423 14:59:46.425210 1 controller.go:512] The operator is healthy I0423 14:59:48.446196 1 insightsuploader.go:146] Nothing to report since 2026-04-23T14:43:16Z I0423 15:01:46.395811 1 configmapobserver.go:84] configmaps "insights-config" not found I0423 15:01:46.421173 1 controller.go:512] The operator is healthy I0423 15:01:46.437205 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0423 15:01:46.442603 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0423 15:01:46.442620 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0423 15:01:46.447699 1 secretconfigobserver.go:119] support secret does not exist I0423 15:03:46.421181 1 controller.go:512] The operator is healthy