I0512 03:02:07.154779 1 cmd.go:253] Using service-serving-cert provided certificates I0512 03:02:07.155224 1 observer_polling.go:159] Starting file observer I0512 03:02:07.594954 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0512 03:02:07.595190 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0512 03:02:07.597249 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0512 03:02:07.597399 1 secure_serving.go:57] Forcing use of http/1.1 only W0512 03:02:07.597428 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0512 03:02:07.597435 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0512 03:02:07.597443 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0512 03:02:07.597445 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0512 03:02:07.597448 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0512 03:02:07.597451 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0512 03:02:07.601384 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0512 03:02:07.601393 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"3fa6bd28-02b2-4dee-a119-013b3e70f47c", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0512 03:02:07.602430 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0512 03:02:07.602446 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0512 03:02:07.602454 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0512 03:02:07.602460 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 03:02:07.602448 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0512 03:02:07.602470 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 03:02:07.602788 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0512 03:02:07.603034 1 secure_serving.go:213] Serving securely on [::]:8443 I0512 03:02:07.603064 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0512 03:02:07.607067 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0512 03:02:07.607087 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 03:02:07.607208 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0512 03:02:07.613739 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:02:07.613759 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0512 03:02:07.618262 1 secretconfigobserver.go:119] support secret does not exist I0512 03:02:07.622816 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:02:07.627332 1 secretconfigobserver.go:119] support secret does not exist I0512 03:02:07.630141 1 recorder.go:161] Pruning old reports every 6h40m33s, max age is 288h0m0s I0512 03:02:07.635090 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0512 03:02:07.635104 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0512 03:02:07.635104 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0512 03:02:07.635115 1 insightsreport.go:296] Starting report retriever I0512 03:02:07.635123 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0512 03:02:07.635147 1 periodic.go:209] Running clusterconfig gatherer I0512 03:02:07.635193 1 tasks_processing.go:45] number of workers: 64 I0512 03:02:07.635212 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 03:02:07.635223 1 tasks_processing.go:69] worker 17 listening for tasks. I0512 03:02:07.635227 1 tasks_processing.go:69] worker 10 listening for tasks. I0512 03:02:07.635229 1 tasks_processing.go:69] worker 11 listening for tasks. I0512 03:02:07.635234 1 tasks_processing.go:69] worker 12 listening for tasks. I0512 03:02:07.635238 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 03:02:07.635240 1 tasks_processing.go:69] worker 13 listening for tasks. I0512 03:02:07.635242 1 tasks_processing.go:71] worker 12 working on ingress task. I0512 03:02:07.635245 1 tasks_processing.go:69] worker 14 listening for tasks. I0512 03:02:07.635246 1 tasks_processing.go:71] worker 13 working on machine_config_pools task. I0512 03:02:07.635249 1 tasks_processing.go:71] worker 14 working on certificate_signing_requests task. I0512 03:02:07.635245 1 tasks_processing.go:69] worker 27 listening for tasks. I0512 03:02:07.635253 1 tasks_processing.go:69] worker 5 listening for tasks. I0512 03:02:07.635262 1 tasks_processing.go:69] worker 18 listening for tasks. I0512 03:02:07.635263 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 03:02:07.635267 1 tasks_processing.go:69] worker 3 listening for tasks. I0512 03:02:07.635273 1 tasks_processing.go:69] worker 4 listening for tasks. I0512 03:02:07.635280 1 tasks_processing.go:69] worker 8 listening for tasks. I0512 03:02:07.635280 1 tasks_processing.go:69] worker 19 listening for tasks. I0512 03:02:07.635280 1 tasks_processing.go:69] worker 7 listening for tasks. I0512 03:02:07.635288 1 tasks_processing.go:69] worker 20 listening for tasks. I0512 03:02:07.635289 1 tasks_processing.go:69] worker 9 listening for tasks. I0512 03:02:07.635298 1 tasks_processing.go:69] worker 24 listening for tasks. I0512 03:02:07.635298 1 tasks_processing.go:69] worker 16 listening for tasks. I0512 03:02:07.635298 1 tasks_processing.go:69] worker 21 listening for tasks. I0512 03:02:07.635299 1 tasks_processing.go:69] worker 15 listening for tasks. I0512 03:02:07.635310 1 tasks_processing.go:69] worker 23 listening for tasks. I0512 03:02:07.635303 1 tasks_processing.go:69] worker 22 listening for tasks. I0512 03:02:07.635315 1 tasks_processing.go:69] worker 37 listening for tasks. I0512 03:02:07.635304 1 tasks_processing.go:69] worker 25 listening for tasks. I0512 03:02:07.635313 1 tasks_processing.go:69] worker 6 listening for tasks. I0512 03:02:07.635324 1 tasks_processing.go:69] worker 33 listening for tasks. I0512 03:02:07.635314 1 tasks_processing.go:69] worker 47 listening for tasks. I0512 03:02:07.635325 1 tasks_processing.go:69] worker 26 listening for tasks. I0512 03:02:07.635333 1 tasks_processing.go:69] worker 34 listening for tasks. I0512 03:02:07.635336 1 tasks_processing.go:69] worker 31 listening for tasks. I0512 03:02:07.635319 1 tasks_processing.go:69] worker 32 listening for tasks. I0512 03:02:07.635338 1 tasks_processing.go:69] worker 36 listening for tasks. I0512 03:02:07.635335 1 tasks_processing.go:69] worker 30 listening for tasks. I0512 03:02:07.635344 1 tasks_processing.go:69] worker 29 listening for tasks. I0512 03:02:07.635347 1 tasks_processing.go:69] worker 45 listening for tasks. I0512 03:02:07.635349 1 tasks_processing.go:69] worker 57 listening for tasks. I0512 03:02:07.635349 1 tasks_processing.go:69] worker 62 listening for tasks. I0512 03:02:07.635352 1 tasks_processing.go:71] worker 17 working on container_images task. I0512 03:02:07.635356 1 tasks_processing.go:69] worker 54 listening for tasks. I0512 03:02:07.635358 1 tasks_processing.go:69] worker 48 listening for tasks. I0512 03:02:07.635359 1 tasks_processing.go:69] worker 55 listening for tasks. I0512 03:02:07.635363 1 tasks_processing.go:69] worker 50 listening for tasks. I0512 03:02:07.635355 1 tasks_processing.go:69] worker 52 listening for tasks. I0512 03:02:07.635366 1 tasks_processing.go:69] worker 53 listening for tasks. I0512 03:02:07.635359 1 tasks_processing.go:69] worker 35 listening for tasks. I0512 03:02:07.635376 1 tasks_processing.go:69] worker 63 listening for tasks. I0512 03:02:07.635332 1 tasks_processing.go:69] worker 42 listening for tasks. I0512 03:02:07.635329 1 tasks_processing.go:69] worker 43 listening for tasks. I0512 03:02:07.635339 1 tasks_processing.go:69] worker 39 listening for tasks. I0512 03:02:07.635339 1 tasks_processing.go:69] worker 44 listening for tasks. I0512 03:02:07.635385 1 tasks_processing.go:71] worker 0 working on image_pruners task. I0512 03:02:07.635380 1 tasks_processing.go:69] worker 49 listening for tasks. I0512 03:02:07.635341 1 tasks_processing.go:69] worker 40 listening for tasks. I0512 03:02:07.635408 1 tasks_processing.go:69] worker 60 listening for tasks. I0512 03:02:07.635348 1 tasks_processing.go:69] worker 41 listening for tasks. I0512 03:02:07.635321 1 tasks_processing.go:69] worker 38 listening for tasks. I0512 03:02:07.635348 1 tasks_processing.go:69] worker 46 listening for tasks. I0512 03:02:07.635360 1 tasks_processing.go:69] worker 56 listening for tasks. I0512 03:02:07.635421 1 tasks_processing.go:69] worker 61 listening for tasks. I0512 03:02:07.635362 1 tasks_processing.go:69] worker 58 listening for tasks. I0512 03:02:07.635326 1 tasks_processing.go:69] worker 28 listening for tasks. I0512 03:02:07.635368 1 tasks_processing.go:71] worker 10 working on silenced_alerts task. I0512 03:02:07.635445 1 tasks_processing.go:71] worker 3 working on infrastructures task. I0512 03:02:07.635451 1 tasks_processing.go:71] worker 27 working on sap_datahubs task. I0512 03:02:07.635456 1 tasks_processing.go:71] worker 28 working on operators_pods_and_events task. I0512 03:02:07.635463 1 tasks_processing.go:71] worker 22 working on sap_config task. I0512 03:02:07.635467 1 tasks_processing.go:71] worker 29 working on metrics task. I0512 03:02:07.635481 1 tasks_processing.go:71] worker 18 working on overlapping_namespace_uids task. I0512 03:02:07.635370 1 tasks_processing.go:69] worker 59 listening for tasks. I0512 03:02:07.635523 1 tasks_processing.go:71] worker 59 working on storage_cluster task. I0512 03:02:07.635525 1 tasks_processing.go:71] worker 2 working on machine_healthchecks task. I0512 03:02:07.635528 1 tasks_processing.go:71] worker 45 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0512 03:02:07.635532 1 tasks_processing.go:71] worker 31 working on machines task. I0512 03:02:07.635538 1 tasks_processing.go:71] worker 47 working on image_registries task. I0512 03:02:07.635541 1 tasks_processing.go:71] worker 36 working on networks task. I0512 03:02:07.635550 1 tasks_processing.go:71] worker 33 working on clusterroles task. I0512 03:02:07.635552 1 tasks_processing.go:71] worker 6 working on pod_network_connectivity_checks task. I0512 03:02:07.635369 1 tasks_processing.go:69] worker 51 listening for tasks. I0512 03:02:07.635371 1 tasks_processing.go:71] worker 11 working on version task. I0512 03:02:07.635615 1 tasks_processing.go:71] worker 8 working on oauths task. I0512 03:02:07.635622 1 tasks_processing.go:71] worker 62 working on feature_gates task. I0512 03:02:07.635624 1 tasks_processing.go:71] worker 61 working on sap_pods task. I0512 03:02:07.635632 1 tasks_processing.go:71] worker 46 working on container_runtime_configs task. I0512 03:02:07.635448 1 tasks_processing.go:71] worker 5 working on olm_operators task. I0512 03:02:07.635656 1 tasks_processing.go:71] worker 52 working on nodenetworkstates task. I0512 03:02:07.635628 1 tasks_processing.go:71] worker 38 working on monitoring_persistent_volumes task. I0512 03:02:07.635725 1 tasks_processing.go:71] worker 23 working on image task. I0512 03:02:07.635616 1 tasks_processing.go:71] worker 40 working on nodes task. I0512 03:02:07.635831 1 tasks_processing.go:71] worker 63 working on storage_classes task. I0512 03:02:07.635842 1 tasks_processing.go:71] worker 42 working on lokistack task. I0512 03:02:07.635852 1 tasks_processing.go:71] worker 54 working on ingress_certificates task. I0512 03:02:07.635869 1 tasks_processing.go:71] worker 55 working on mutating_webhook_configurations task. I0512 03:02:07.635243 1 tasks_processing.go:71] worker 1 working on install_plans task. I0512 03:02:07.635530 1 tasks_processing.go:71] worker 37 working on openstack_version task. I0512 03:02:07.635624 1 tasks_processing.go:71] worker 56 working on support_secret task. I0512 03:02:07.635589 1 tasks_processing.go:71] worker 32 working on config_maps task. I0512 03:02:07.635534 1 tasks_processing.go:71] worker 25 working on openstack_dataplanenodesets task. I0512 03:02:07.635525 1 tasks_processing.go:71] worker 26 working on crds task. I0512 03:02:07.635834 1 tasks_processing.go:71] worker 58 working on pdbs task. I0512 03:02:07.635601 1 tasks_processing.go:71] worker 57 working on openshift_machine_api_events task. I0512 03:02:07.635599 1 tasks_processing.go:71] worker 51 working on dvo_metrics task. I0512 03:02:07.635604 1 tasks_processing.go:71] worker 4 working on schedulers task. I0512 03:02:07.635605 1 tasks_processing.go:71] worker 39 working on nodenetworkconfigurationpolicies task. I0512 03:02:07.635610 1 tasks_processing.go:71] worker 44 working on proxies task. I0512 03:02:07.635610 1 tasks_processing.go:71] worker 19 working on machine_sets task. I0512 03:02:07.635613 1 tasks_processing.go:71] worker 49 working on validating_webhook_configurations task. I0512 03:02:07.635612 1 tasks_processing.go:71] worker 7 working on qemu_kubevirt_launcher_logs task. I0512 03:02:07.635610 1 tasks_processing.go:71] worker 43 working on active_alerts task. I0512 03:02:07.635618 1 tasks_processing.go:71] worker 60 working on authentication task. I0512 03:02:07.635617 1 tasks_processing.go:71] worker 20 working on openshift_logging task. I0512 03:02:07.635621 1 tasks_processing.go:71] worker 41 working on openstack_controlplanes task. I0512 03:02:07.635707 1 tasks_processing.go:71] worker 24 working on cost_management_metrics_configs task. I0512 03:02:07.635712 1 tasks_processing.go:71] worker 16 working on cluster_apiserver task. I0512 03:02:07.635717 1 tasks_processing.go:71] worker 21 working on machine_autoscalers task. I0512 03:02:07.635721 1 tasks_processing.go:71] worker 15 working on machine_configs task. I0512 03:02:07.635822 1 tasks_processing.go:71] worker 53 working on jaegers task. I0512 03:02:07.635827 1 tasks_processing.go:71] worker 35 working on tsdb_status task. I0512 03:02:07.635919 1 tasks_processing.go:71] worker 48 working on operators task. I0512 03:02:07.636274 1 tasks_processing.go:71] worker 50 working on service_accounts task. I0512 03:02:07.635601 1 tasks_processing.go:71] worker 9 working on node_logs task. I0512 03:02:07.635595 1 tasks_processing.go:71] worker 30 working on openstack_dataplanedeployments task. I0512 03:02:07.635525 1 tasks_processing.go:71] worker 34 working on aggregated_monitoring_cr_names task. E0512 03:02:07.641552 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:40629->172.30.0.10:53: read: connection refused I0512 03:02:07.641573 1 tasks_processing.go:71] worker 35 working on ceph_cluster task. E0512 03:02:07.641567 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:40629->172.30.0.10:53: read: connection refused E0512 03:02:07.641586 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:40629->172.30.0.10:53: read: connection refused I0512 03:02:07.641596 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 2.928358ms to process 0 records E0512 03:02:07.641606 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:40629->172.30.0.10:53: read: connection refused I0512 03:02:07.641617 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 6.103297ms to process 0 records I0512 03:02:07.641632 1 tasks_processing.go:74] worker 29 stopped. E0512 03:02:07.641689 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:39596->172.30.0.10:53: read: connection refused I0512 03:02:07.641705 1 tasks_processing.go:74] worker 43 stopped. E0512 03:02:07.641712 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:39596->172.30.0.10:53: read: connection refused I0512 03:02:07.641721 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 3.852447ms to process 0 records E0512 03:02:07.641748 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:39596->172.30.0.10:53: read: connection refused I0512 03:02:07.641773 1 tasks_processing.go:74] worker 10 stopped. E0512 03:02:07.641782 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.128.0.21:39596->172.30.0.10:53: read: connection refused I0512 03:02:07.641792 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 6.331074ms to process 0 records I0512 03:02:07.642344 1 tasks_processing.go:74] worker 27 stopped. I0512 03:02:07.642357 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 6.885263ms to process 0 records I0512 03:02:07.642366 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 6.490769ms to process 0 records I0512 03:02:07.642372 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 6.730071ms to process 0 records I0512 03:02:07.642376 1 tasks_processing.go:74] worker 42 stopped. I0512 03:02:07.642381 1 tasks_processing.go:74] worker 46 stopped. I0512 03:02:07.642405 1 tasks_processing.go:74] worker 6 stopped. E0512 03:02:07.642420 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0512 03:02:07.642431 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 6.841316ms to process 0 records I0512 03:02:07.642447 1 tasks_processing.go:74] worker 52 stopped. I0512 03:02:07.642460 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 6.780229ms to process 0 records I0512 03:02:07.642467 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 6.303857ms to process 0 records I0512 03:02:07.642473 1 tasks_processing.go:74] worker 37 stopped. I0512 03:02:07.642481 1 tasks_processing.go:74] worker 2 stopped. E0512 03:02:07.642489 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0512 03:02:07.642496 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 6.94566ms to process 0 records I0512 03:02:07.642704 1 tasks_processing.go:74] worker 12 stopped. I0512 03:02:07.642913 1 recorder.go:75] Recording config/ingress with fingerprint=b70214ffb2141edcf45afdbe6662065dad9c5795d1ae6fda1bb5a789504600fe I0512 03:02:07.642929 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 7.450066ms to process 1 records I0512 03:02:07.643008 1 tasks_processing.go:74] worker 0 stopped. I0512 03:02:07.643101 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=af09ea18aa876f130c7b6335863e89386a30c44602c5b754ceff1fdb5213758d I0512 03:02:07.643109 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 7.307121ms to process 1 records I0512 03:02:07.646349 1 tasks_processing.go:74] worker 8 stopped. I0512 03:02:07.646554 1 recorder.go:75] Recording config/oauth with fingerprint=26b3febf81e73040bf48badd9c76277f0034fd0782820cb0693641894a10c6d5 I0512 03:02:07.646567 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 10.715602ms to process 1 records I0512 03:02:07.656387 1 tasks_processing.go:74] worker 22 stopped. I0512 03:02:07.656402 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 20.91593ms to process 0 records E0512 03:02:07.656410 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0512 03:02:07.656417 1 gather.go:177] gatherer "clusterconfig" function "machines" took 20.867902ms to process 0 records I0512 03:02:07.656423 1 tasks_processing.go:74] worker 31 stopped. I0512 03:02:07.656459 1 tasks_processing.go:74] worker 3 stopped. I0512 03:02:07.657087 1 recorder.go:75] Recording config/infrastructure with fingerprint=f0979a4cbea7d341dee023004a5b8f23dc9ecb0cca2da0110dddbbbf629e9d18 I0512 03:02:07.657105 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 21.00151ms to process 1 records I0512 03:02:07.657116 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 20.063845ms to process 0 records I0512 03:02:07.657131 1 tasks_processing.go:74] worker 25 stopped. I0512 03:02:07.657240 1 recorder.go:75] Recording config/featuregate with fingerprint=246269b36cae33fd7a55448dfca1438b52c10b9f22ce0a3b00c1cabdd71a83a7 I0512 03:02:07.657253 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 20.972694ms to process 1 records I0512 03:02:07.657262 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 20.986642ms to process 0 records I0512 03:02:07.657265 1 tasks_processing.go:74] worker 62 stopped. I0512 03:02:07.657267 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 20.902179ms to process 0 records I0512 03:02:07.657270 1 tasks_processing.go:74] worker 61 stopped. I0512 03:02:07.657273 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 19.336325ms to process 0 records I0512 03:02:07.657274 1 tasks_processing.go:74] worker 38 stopped. I0512 03:02:07.657279 1 tasks_processing.go:74] worker 39 stopped. I0512 03:02:07.657345 1 tasks_processing.go:74] worker 23 stopped. I0512 03:02:07.657366 1 recorder.go:75] Recording config/image with fingerprint=11200121b8f93083a63275f9b42b56adb6a6641ff22e3079a9a1f1758df799a7 I0512 03:02:07.657376 1 gather.go:177] gatherer "clusterconfig" function "image" took 20.936486ms to process 1 records I0512 03:02:07.657387 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 21.478906ms to process 0 records I0512 03:02:07.657388 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0512 03:02:07.657395 1 tasks_processing.go:74] worker 13 stopped. I0512 03:02:07.657402 1 controller.go:326] The initial operator extension status is healthy I0512 03:02:07.657411 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 03:02:07.657416 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0512 03:02:07.657418 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0512 03:02:07.657433 1 controller.go:489] The operator is still being initialized I0512 03:02:07.657439 1 controller.go:512] The operator is healthy I0512 03:02:07.657471 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=4aafd6c97718c0d2921295b6ac96a827f0076c2fcd7e63276b3faf68b755897a I0512 03:02:07.657490 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=aed2f31c53e13ec01ae01cdd09f4a6b98a1c56e68b61855b594ccbf4dbddf639 I0512 03:02:07.657491 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0512 03:02:07.657500 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 21.326732ms to process 2 records E0512 03:02:07.657510 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0512 03:02:07.657518 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 20.961641ms to process 0 records I0512 03:02:07.657543 1 tasks_processing.go:74] worker 56 stopped. I0512 03:02:07.657543 1 tasks_processing.go:74] worker 63 stopped. I0512 03:02:07.657555 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0512 03:02:07.657618 1 tasks_processing.go:74] worker 36 stopped. W0512 03:02:07.657644 1 operator.go:288] started I0512 03:02:07.657649 1 recorder.go:75] Recording config/network with fingerprint=0dc157da3d7ea18db6c3a39402b00f11d708c3cdd592966cb0617a48924f137b I0512 03:02:07.657664 1 gather.go:177] gatherer "clusterconfig" function "networks" took 21.809349ms to process 1 records I0512 03:02:07.657675 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 20.49077ms to process 0 records I0512 03:02:07.657666 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0512 03:02:07.657686 1 tasks_processing.go:74] worker 57 stopped. I0512 03:02:07.657726 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=e94a73521974a6c9c118239ce59ab88cf44ba256016a60e270d61cab3930b7b9 I0512 03:02:07.657736 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 20.182681ms to process 1 records I0512 03:02:07.657740 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 22.045621ms to process 0 records I0512 03:02:07.657750 1 tasks_processing.go:74] worker 4 stopped. I0512 03:02:07.657759 1 tasks_processing.go:74] worker 59 stopped. I0512 03:02:07.657792 1 recorder.go:75] Recording config/proxy with fingerprint=73bcdaa3948e211d27fb43d934a6a1f7434c747cc13897135ad8fb80f007fe56 I0512 03:02:07.657798 1 tasks_processing.go:74] worker 44 stopped. I0512 03:02:07.657800 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 20.168117ms to process 1 records I0512 03:02:07.658157 1 tasks_processing.go:74] worker 47 stopped. I0512 03:02:07.658454 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=e2eaa5bd480bc22b3b94285e2ed10e2adcbb987abff4a0c3b240bc2e93915d87 I0512 03:02:07.658466 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 22.60597ms to process 1 records I0512 03:02:07.661217 1 tasks_processing.go:74] worker 18 stopped. I0512 03:02:07.661243 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0512 03:02:07.661252 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 25.723559ms to process 1 records I0512 03:02:07.662079 1 gather_logs.go:145] no pods in namespace were found I0512 03:02:07.662088 1 tasks_processing.go:74] worker 7 stopped. I0512 03:02:07.662094 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 24.38631ms to process 0 records I0512 03:02:07.662190 1 tasks_processing.go:74] worker 16 stopped. I0512 03:02:07.662329 1 recorder.go:75] Recording config/apiserver with fingerprint=5301470ad18f5b380bc4c331e643f44f64aa336773060f9d4bd62b49fda01849 I0512 03:02:07.662342 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 23.773962ms to process 1 records I0512 03:02:07.662447 1 tasks_processing.go:74] worker 60 stopped. I0512 03:02:07.662698 1 recorder.go:75] Recording config/authentication with fingerprint=41ddd67646fd0820cb58cca281fc8dbb27d8a0323a6426470309db776987dc50 I0512 03:02:07.662720 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 24.555585ms to process 1 records W0512 03:02:07.665541 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 03:02:07.667166 1 tasks_processing.go:74] worker 19 stopped. I0512 03:02:07.667181 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 29.602386ms to process 0 records I0512 03:02:07.667263 1 tasks_processing.go:74] worker 21 stopped. I0512 03:02:07.667279 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 28.804943ms to process 0 records I0512 03:02:07.667565 1 tasks_processing.go:74] worker 40 stopped. I0512 03:02:07.667851 1 recorder.go:75] Recording config/node/ip-10-0-1-251.ec2.internal with fingerprint=34c24484ffdc66940ff133a7eb9d38665677f288af453389261dc238433ef500 I0512 03:02:07.667862 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 31.726649ms to process 1 records I0512 03:02:07.667954 1 tasks_processing.go:74] worker 41 stopped. I0512 03:02:07.667973 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 29.700832ms to process 0 records I0512 03:02:07.668039 1 tasks_processing.go:74] worker 58 stopped. I0512 03:02:07.668128 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=a07e65893d4836bef545494d79f5dff8f2007f81c95c3cc44940ca51e6251f47 I0512 03:02:07.668148 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=19c44c94a9fb72a54aab34e5686dcc5317e6d573634bbdfe1650adb75278a1b8 I0512 03:02:07.668160 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=0ddc01a783798b940578eb4154f438622a17651ea7e9b152029d75939804acef I0512 03:02:07.668174 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=9f4ac352b70d19f4505bb37e57a3930133022320eeb2dd427b63f7c4f9a02669 I0512 03:02:07.668186 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=4aa3325c09fa6fba763fd4d7986001c75fdb88ff934b4a3b0128c59489067361 I0512 03:02:07.668200 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=9bc6fafef76aedb9cd14a1dd544ed68840609de4dfe9f546b9371532f36719d7 I0512 03:02:07.668205 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 31.449751ms to process 6 records I0512 03:02:07.668236 1 tasks_processing.go:74] worker 20 stopped. I0512 03:02:07.668254 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 30.13842ms to process 0 records I0512 03:02:07.668349 1 tasks_processing.go:74] worker 55 stopped. I0512 03:02:07.668588 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=dac8e32dd12809147864952b06a546fd968b7f71406a7cfca4007d2fd65ff3db I0512 03:02:07.668650 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=556a9a8bcdbca3220fe7c444e413a02866b66798ab9c6672e25580b32ddc0be0 I0512 03:02:07.668716 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=0d49a29b97b50138f7f4db339081c68b88665312ba30425b057072646a5f963f I0512 03:02:07.668729 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 32.471488ms to process 3 records I0512 03:02:07.668746 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 30.146051ms to process 0 records I0512 03:02:07.668755 1 tasks_processing.go:74] worker 24 stopped. I0512 03:02:07.668769 1 tasks_processing.go:74] worker 49 stopped. I0512 03:02:07.668867 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=b75226decd84b440e19e58cf5864ab690744bb9a0ead00534a4ec26676917681 I0512 03:02:07.668966 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=001964af52341736daf9f57fed2e69d7f6d80eeef4cf2d29ec77164f41509564 I0512 03:02:07.668993 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=03cd09a0a86c226ea8ce1260f65f8a3b0234083c932aa07151385183fb2cac9d I0512 03:02:07.669022 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=a06b555e283c1739cf14437062dd089aaa0b2e7a185683d7b9c525e27834b123 I0512 03:02:07.669058 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=1de45170aa59e425257821fad4058b7dde93826c392d0dab61529b95ffbf9c92 I0512 03:02:07.669087 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=316167bb9d7ee12dd9c81e9ee51de13ce140f19f6b39e926503bfa3235f514c9 I0512 03:02:07.669116 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=6c8e23b2b8fe2804e347e27d395ce9dddbdd78456af3e154704600c51232de3e I0512 03:02:07.669154 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=edc4c86ff7563f860897486b1b47b1717c34049450957217d1fe1553d624319a I0512 03:02:07.669187 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=f47f92e9dfebf2bec7b4f8e08b16af9b75002ec57f7673730a11ad097c8bad94 I0512 03:02:07.669216 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=a9881ca6116101403200d29c728bfbc517aba6c96b46b569f919dd2b7c0b916d I0512 03:02:07.669244 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=b2727eb14d9c29d4351cac8ea45724200507b62446a90699d4f3c010e1169aac I0512 03:02:07.669252 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 31.151717ms to process 11 records I0512 03:02:07.669263 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 33.572882ms to process 0 records I0512 03:02:07.669269 1 tasks_processing.go:74] worker 14 stopped. I0512 03:02:07.672538 1 tasks_processing.go:74] worker 30 stopped. I0512 03:02:07.672549 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 33.661769ms to process 0 records I0512 03:02:07.673007 1 tasks_processing.go:74] worker 53 stopped. I0512 03:02:07.673070 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 34.424428ms to process 0 records I0512 03:02:07.673322 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0512 03:02:07.673379 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0512 03:02:07.673406 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0512 03:02:07.673432 1 controller.go:212] Source scaController *sca.Controller is not ready I0512 03:02:07.673467 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0512 03:02:07.673546 1 controller.go:489] The operator is still being initialized I0512 03:02:07.673574 1 controller.go:512] The operator is healthy I0512 03:02:07.674094 1 tasks_processing.go:74] worker 35 stopped. I0512 03:02:07.674115 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 32.512302ms to process 0 records I0512 03:02:07.677131 1 tasks_processing.go:74] worker 17 stopped. I0512 03:02:07.678388 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq with fingerprint=1884132251a6c29be2a87b096534a4f1502525db4c22fe31ccab4a6fc3afbc2c I0512 03:02:07.678474 1 recorder.go:75] Recording config/running_containers with fingerprint=c1b81173e947cbc3bfdc298690df41e08a6ff6767cdc69a3743a3f464f84f80b I0512 03:02:07.678489 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 41.747151ms to process 2 records I0512 03:02:07.678574 1 tasks_processing.go:74] worker 33 stopped. I0512 03:02:07.678661 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=31e39b58f46a3d645b4d6b8c0d1f5243810d8dd2d4cd6f656875e9840ddd597d I0512 03:02:07.678773 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=38e36a48c2474af4bbf9f94679aa817084f625dc9498f1cc1075598353f3c146 I0512 03:02:07.678786 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 41.889355ms to process 2 records I0512 03:02:07.684684 1 tasks_processing.go:74] worker 11 stopped. I0512 03:02:07.686253 1 recorder.go:75] Recording config/version with fingerprint=48698e76fd765c1bfcc5b8fd1eff75ace19ebd9fb453cc1042207b173a9adc55 I0512 03:02:07.686333 1 recorder.go:75] Recording config/id with fingerprint=72697f43ea81c79b7aebb83faca0d2e9f3472c2bd4dd9a09f855ed72b7e969a3 I0512 03:02:07.686366 1 gather.go:177] gatherer "clusterconfig" function "version" took 49.063959ms to process 2 records I0512 03:02:07.686408 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 45.978188ms to process 0 records I0512 03:02:07.686454 1 tasks_processing.go:74] worker 5 stopped. I0512 03:02:07.686488 1 tasks_processing.go:74] worker 9 stopped. I0512 03:02:07.686557 1 recorder.go:75] Recording config/olm_operators with fingerprint=b9514cee28614f07b3cdafbf9be7e7cddbb50f457f35901d5093968db9fc07ac I0512 03:02:07.686580 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 50.675954ms to process 1 records I0512 03:02:07.686589 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 50.811549ms to process 0 records I0512 03:02:07.686599 1 tasks_processing.go:74] worker 45 stopped. I0512 03:02:07.687652 1 tasks_processing.go:74] worker 26 stopped. I0512 03:02:07.688312 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=9c1e58f9fcce708df2c5d7cc6781be03b4754fb9ce503a897e069dae76599b62 I0512 03:02:07.688567 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=eece8486e0aeb59b798d3f7a082df6949de4a7ef69f9c59ce0f8c2d8e700976f I0512 03:02:07.688584 1 gather.go:177] gatherer "clusterconfig" function "crds" took 51.111829ms to process 2 records E0512 03:02:07.689062 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2768873eb4-3a55-47cc-8a47-664a767e9907%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:58295->172.30.0.10:53: read: connection refused I0512 03:02:07.689076 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2768873eb4-3a55-47cc-8a47-664a767e9907%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:58295->172.30.0.10:53: read: connection refused I0512 03:02:07.690672 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0512 03:02:07.692134 1 tasks_processing.go:74] worker 15 stopped. I0512 03:02:07.692159 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0512 03:02:07.692168 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 53.604974ms to process 1 records W0512 03:02:07.693518 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:41007->172.30.0.10:53: read: connection refused I0512 03:02:07.693529 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.21:41007->172.30.0.10:53: read: connection refused I0512 03:02:07.698412 1 tasks_processing.go:74] worker 34 stopped. I0512 03:02:07.698427 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 58.833446ms to process 0 records I0512 03:02:07.701948 1 tasks_processing.go:74] worker 54 stopped. E0512 03:02:07.701961 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0512 03:02:07.701967 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q849d8j6fs4l0vid19bea5pdr4lvpfb-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q849d8j6fs4l0vid19bea5pdr4lvpfb-primary-cert-bundle-secret" not found I0512 03:02:07.702027 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=a3a42acdcd81eed66e551dfd437d9a2d3ce512f13d23712f44c91214d47a75ed I0512 03:02:07.702039 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 66.086484ms to process 1 records I0512 03:02:07.702987 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0512 03:02:07.702994 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0512 03:02:07.703016 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0512 03:02:07.707497 1 base_controller.go:82] Caches are synced for ConfigController I0512 03:02:07.707508 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0512 03:02:07.711065 1 tasks_processing.go:74] worker 32 stopped. E0512 03:02:07.711077 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0512 03:02:07.711085 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0512 03:02:07.711089 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0512 03:02:07.711099 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0512 03:02:07.711124 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0512 03:02:07.711132 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0512 03:02:07.711136 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0512 03:02:07.711145 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0512 03:02:07.711183 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0512 03:02:07.711191 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0512 03:02:07.711196 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 74.742499ms to process 7 records I0512 03:02:07.713950 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 03:02:07.757859 1 base_controller.go:82] Caches are synced for LoggingSyncer I0512 03:02:07.757877 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0512 03:02:08.666522 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 03:02:09.270999 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 19 containers I0512 03:02:09.271013 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1324517 bytes I0512 03:02:09.271218 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-bbcgr pod in namespace openshift-network-console (previous: false). I0512 03:02:09.476458 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-6ddbfdf749-w6jlk pod in namespace openshift-network-console (previous: false). I0512 03:02:09.503954 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0512 03:02:09.666534 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 03:02:09.676526 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-f2cgq pod in namespace openshift-console-operator (previous: true). I0512 03:02:09.876449 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-575cd97545-f2cgq pod in namespace openshift-console-operator (previous: false). I0512 03:02:10.080031 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-77485996b6-hxc4g pod in namespace openshift-console (previous: false). I0512 03:02:10.106364 1 tasks_processing.go:74] worker 48 stopped. I0512 03:02:10.106484 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=1032bc7f60b6f6920b0f478fd7a6cf2e3fb6c7666a9b2c3fa0aa39ca62332427 I0512 03:02:10.106533 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=b1d3a57201f472fba248e6e22732d03f4ebac77f3d7204455295e9c6f31f376a I0512 03:02:10.106558 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=9321990725aa5bd2b23504b122d786b7a2a7367881fd73d48c91d30ba8d300f5 I0512 03:02:10.106585 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0512 03:02:10.106608 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=ca25a498e3277c4e13bba156f87569c3b038261ccf74676ef98d82af606805c3 I0512 03:02:10.106631 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0512 03:02:10.106656 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=e62f2cfeea03f5eaebbb617a4c89848fe6a7ec643dc3665636092657c691729f I0512 03:02:10.106686 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=c302bdee6a0b24878767ccb12d03e135bbe290e9fc94f7b25022dbca658306b9 I0512 03:02:10.106709 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=a648c51523dcedb512a560dbbfff7e4ce2306cd72ea57b7f6a5170100c4d2e5c I0512 03:02:10.106739 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=cf9ec5cee7bb416a96bc91584bc15902e7fdc3cdfc88862dd6104375a08d4c5f I0512 03:02:10.107017 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=5ca572956c57cd321b8a5d639b4d413195f4d21b10fd8e325af2f381b9dc04f3 I0512 03:02:10.107038 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=88a8f5a896422dd2afa49a8df8bf03fddf875870c0e208f9d599f651faaa6607 I0512 03:02:10.107046 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0512 03:02:10.107063 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=7fbe8aa8df22de333f3ab12e28b911ae5091b9c1d0b945eb6a9face1c4028297 I0512 03:02:10.107073 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0512 03:02:10.107090 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=434eade6a959d9f90b3b3f7ff844845d22fdcaa0bfb175b451e3f9a7015e51a8 I0512 03:02:10.107099 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0512 03:02:10.107117 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=1cf28ee6c8641ab82234a19903846542b0f493c5a0174df2fdb03ba96e9c055f I0512 03:02:10.107127 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0512 03:02:10.107146 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=2d004a1b52f9f4e8847e0cd65c325ae310e08d7701f010be69790607ad4c2f64 I0512 03:02:10.107246 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=b7ba5fcd326b0169d4b99e704775117bf9af4555af323ca97ab4ea0e373af7a1 I0512 03:02:10.107257 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0512 03:02:10.107263 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0512 03:02:10.107284 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0512 03:02:10.107304 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=bf5ca46b1a837c66dca95b3130807f40b913ff767a5603d52d0095de54d452e7 I0512 03:02:10.107327 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=af42582a81b010c33ead057c2bc036de3b54e65944ff6278ec819084fc1a7fc6 I0512 03:02:10.107336 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0512 03:02:10.107350 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=6da74f4396df74f1a00ea7bd1b8b017d20bb93af0537dacdfd70f2855963b2c3 I0512 03:02:10.107360 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0512 03:02:10.107373 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=549fc306f1669cac027bb972948d343e05d2ccc6ac8001975398e45de15f7da7 I0512 03:02:10.107387 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=c3383ae633140406787d0a71da98576e17e2a9b303e382c1e91cf6ec0c0ce7a1 I0512 03:02:10.107402 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=03c71af51ad115f0a56fd6aadc5e376e1df261271ee819627fcc3f174f565cc8 I0512 03:02:10.107419 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=b7212d6a0edd8cea25e242503bc5e8a639597f494d999598705538c0e2e6f511 I0512 03:02:10.107440 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=ced8d5a15942bbcf788c1c6876853ba462aa08a42821f115d7e573581832ff70 I0512 03:02:10.107449 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0512 03:02:10.107476 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=1cca18e5a0a45e0b8b5f98e6230a7e6460e985d67e9f326d63aba23ae2a61e11 I0512 03:02:10.107501 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0512 03:02:10.107511 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0512 03:02:10.107518 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.467696647s to process 38 records I0512 03:02:10.269874 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:10.269905 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-77485996b6-wh84m pod in namespace openshift-console (previous: false). I0512 03:02:10.469606 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:10.469624 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-7cdbb89-kqnkk pod in namespace openshift-console (previous: false). W0512 03:02:10.667821 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 03:02:10.676271 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-mp98s pod in namespace openshift-console (previous: false). I0512 03:02:10.871100 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:10.871116 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-56cb7c9577-vp6ct pod in namespace openshift-console (previous: false). I0512 03:02:11.075490 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-th74h pod in namespace openshift-dns (previous: false). I0512 03:02:11.276297 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-th74h pod in namespace openshift-dns (previous: false). I0512 03:02:11.477433 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-mnlbd pod in namespace openshift-dns (previous: false). W0512 03:02:11.665720 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0512 03:02:11.678664 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:11.678678 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-67596fb5f5-mrcdm pod in namespace openshift-image-registry (previous: false). I0512 03:02:11.931290 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-67979f7d95-ss6dl pod in namespace openshift-image-registry (previous: false). I0512 03:02:12.076144 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:12.076162 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-67979f7d95-t4lqj pod in namespace openshift-image-registry (previous: false). I0512 03:02:12.270993 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:12.271010 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-5b94h pod in namespace openshift-image-registry (previous: false). I0512 03:02:12.477378 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-548d975bbb-gl56p pod in namespace openshift-ingress (previous: false). W0512 03:02:12.666032 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0512 03:02:12.666056 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 03:02:12.666072 1 tasks_processing.go:74] worker 51 stopped. E0512 03:02:12.666083 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0512 03:02:12.666094 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0512 03:02:12.666105 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.028936247s to process 1 records I0512 03:02:12.677602 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-548d975bbb-t5ggc pod in namespace openshift-ingress (previous: false). I0512 03:02:12.870521 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0512 03:02:12.870540 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5bdf876667-s6bb5 pod in namespace openshift-ingress (previous: false). I0512 03:02:13.077284 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-htvcp pod in namespace openshift-ingress-canary (previous: false). I0512 03:02:13.276509 1 tasks_processing.go:74] worker 28 stopped. I0512 03:02:13.276608 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=86cdbcd5ef8b92ffc83e9ef25ee6b7980d3fa2009005839668f66b40fd704734 I0512 03:02:13.276734 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=95ea5d894dcf57bb9bb9f0532b2422d4542e03fa5d2f87aa03d2d99ab4188891 I0512 03:02:13.276782 1 recorder.go:75] Recording events/openshift-console with fingerprint=792763c53de47a7908c6b97ebd7542740ff6e8aeb77f5f716a0bd0571e7731e8 I0512 03:02:13.276807 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=5d7d8e123680f2225d9525362934359d69e40e8b35262884ea652adc16d53767 I0512 03:02:13.276837 1 recorder.go:75] Recording events/openshift-dns with fingerprint=a9d149ec8557a6c3bb60a7269bcca7347ae34d56942c3957e60db12a19102e54 I0512 03:02:13.276958 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=a383c3c6bff50783752181e1763646e379a0cbe79e9a1008c5d863a8dce06197 I0512 03:02:13.276985 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=658b6c59754a164c1a3057541af316027d5da518f26644576fb1f31cb091cfb9 I0512 03:02:13.277044 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=1b4314241bd11e276b18ca32f75773ae46a925357b997a38b26e06d0729d1050 I0512 03:02:13.277063 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=1c689863ccda4777ecff385de91f5048244e1498856c05a9d854bed71227ec3a I0512 03:02:13.277070 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-bbcgr/networking-console-plugin_current.log with fingerprint=3ac66056a0c0bff182c7f2496fbee04f9b134d09cb0b8d3ef4c66d3fd32d96aa I0512 03:02:13.277078 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-6ddbfdf749-w6jlk/networking-console-plugin_current.log with fingerprint=d3e135e0aa6cb102d6a277651b4f93e47387ff5e94d24d2bfbce7093c72d1ad0 I0512 03:02:13.277209 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq with fingerprint=1884132251a6c29be2a87b096534a4f1502525db4c22fe31ccab4a6fc3afbc2c E0512 03:02:13.277228 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json" was already recorded and had the fingerprint "1884132251a6c29be2a87b096534a4f1502525db4c22fe31ccab4a6fc3afbc2c", overwriting with the record having fingerprint "1884132251a6c29be2a87b096534a4f1502525db4c22fe31ccab4a6fc3afbc2c" W0512 03:02:13.277239 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json" because of the warning: warning: the record with the same fingerprint "1884132251a6c29be2a87b096534a4f1502525db4c22fe31ccab4a6fc3afbc2c" was already recorded at path "config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json" I0512 03:02:13.277250 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-f2cgq/console-operator_previous.log with fingerprint=214e762c29e265fbf349e0a35d6854b1a73736aa4e94ea04d08edbcf8f0f64dc I0512 03:02:13.277422 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-575cd97545-f2cgq/console-operator_current.log with fingerprint=e208451288858e5760f8e28235930f70c9bf66680231f1960cc3b8a9dacfd1de I0512 03:02:13.277435 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-7cdbb89-kqnkk/console_current.log with fingerprint=2b5ecc0b5dabfc464ebfc8b2e52966fc140aed197f8f1be4158292109695a371 I0512 03:02:13.277441 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-56cb7c9577-vp6ct/download-server_current.log with fingerprint=edc0ae3502eddff7f53e0af7c8585cd98b9e7ddbb7ff1851456b0173738957d1 I0512 03:02:13.277447 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-th74h/dns_current.log with fingerprint=88f1cbb4a70f1fed761960b4ef68b42e56b2b827fb171eb708bc5e26c8c90660 I0512 03:02:13.277455 1 recorder.go:75] Recording config/pod/openshift-dns/logs/dns-default-th74h/kube-rbac-proxy_current.log with fingerprint=f3afa79d9682eeb4c5f6f3eb5e787158968804752c49b848381b86002038942c I0512 03:02:13.277467 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-67596fb5f5-mrcdm/registry_current.log with fingerprint=e03d4a07dcf5d0122fce2fbdc96ca5c884bcfdcade3519405c87da3c96c8b66f I0512 03:02:13.277473 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-5b94h/node-ca_current.log with fingerprint=80f34a39c51701b1d362c318d7077c2132a2d90421823397b447d6aac103f704 I0512 03:02:13.277491 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-548d975bbb-gl56p/router_current.log with fingerprint=abac27e25b0d5b443e683beb86d2c434148632acd547fd7908f71bb3d82e1c5e I0512 03:02:13.277532 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-5bdf876667-s6bb5/router_current.log with fingerprint=83ea76ff07feea93bd95fc62584f947a5319952f7f19f4318eabed38847a8e21 I0512 03:02:13.277538 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-htvcp/serve-healthcheck-canary_current.log with fingerprint=58d2b13be742715eee4aecbc6b066546ac31c3366e40de2de408f42ecca504eb I0512 03:02:13.277543 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 5.641036825s to process 23 records I0512 03:02:20.677181 1 tasks_processing.go:74] worker 1 stopped. I0512 03:02:20.677221 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0512 03:02:20.677232 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.041202405s to process 1 records I0512 03:02:21.446815 1 tasks_processing.go:74] worker 50 stopped. I0512 03:02:21.447123 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=bae76059ad12f92c03c06b36414d832a0dbbce27d94c635a80399044861a2efd I0512 03:02:21.447140 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.80804382s to process 1 records E0512 03:02:21.447202 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.812s with: function \"tsdb_status\" failed with an error, function \"metrics\" failed with an error, function \"active_alerts\" failed with an error, function \"silenced_alerts\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"support_secret\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json\"" I0512 03:02:21.448307 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "tsdb_status" failed with an error, function "metrics" failed with an error, function "active_alerts" failed with an error, function "silenced_alerts" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "support_secret" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-575cd97545-f2cgq.json" I0512 03:02:21.448322 1 periodic.go:209] Running workloads gatherer I0512 03:02:21.448337 1 tasks_processing.go:45] number of workers: 2 I0512 03:02:21.448348 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 03:02:21.448350 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 03:02:21.448360 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0512 03:02:21.448352 1 tasks_processing.go:71] worker 1 working on workload_info task. I0512 03:02:21.454638 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-251.ec2.internal... I0512 03:02:21.476428 1 tasks_processing.go:74] worker 0 stopped. I0512 03:02:21.476446 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 28.052888ms to process 0 records I0512 03:02:25.060314 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 4s I0512 03:02:25.078822 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 21s for image data I0512 03:02:25.088094 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0512 03:02:25.097296 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (9ms) I0512 03:02:25.106367 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0512 03:02:25.115972 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (10ms) I0512 03:02:25.125495 1 gather_workloads_info.go:387] No image sha256:a505dd5b62d744c2cfda2666b5e21e0a4704354a3114488bd3291456085e1e33 (9ms) I0512 03:02:25.134337 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (9ms) I0512 03:02:25.144992 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (11ms) I0512 03:02:25.154759 1 gather_workloads_info.go:387] No image sha256:fab570ecdc9c3620b032bc4c5b44240f5ebac0bb9e48dfeea32bcd36eb462e5a (10ms) I0512 03:02:25.164818 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (10ms) I0512 03:02:25.173705 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (9ms) I0512 03:02:25.191105 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (17ms) I0512 03:02:25.288680 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (98ms) I0512 03:02:25.388950 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0512 03:02:25.488482 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0512 03:02:25.588513 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0512 03:02:25.688975 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0512 03:02:25.788220 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (99ms) I0512 03:02:25.888560 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0512 03:02:25.989003 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (100ms) I0512 03:02:26.088041 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (99ms) I0512 03:02:26.188294 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (100ms) I0512 03:02:26.288292 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0512 03:02:26.388191 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0512 03:02:26.490016 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (102ms) I0512 03:02:26.588231 1 gather_workloads_info.go:387] No image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (98ms) I0512 03:02:26.688978 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0512 03:02:26.787784 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0512 03:02:26.888338 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (101ms) I0512 03:02:26.988437 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (100ms) I0512 03:02:27.088660 1 gather_workloads_info.go:387] No image sha256:ef2bfb546cd8aac31b7671585f718f44b268ed43a9c8562016ab741558a7dfa1 (100ms) I0512 03:02:27.188536 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0512 03:02:27.288451 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0512 03:02:27.388191 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0512 03:02:27.488328 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0512 03:02:27.588048 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (100ms) I0512 03:02:27.688739 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (101ms) I0512 03:02:27.787340 1 gather_workloads_info.go:387] No image sha256:d3209a996e5ce362136e70a0bcf06d89973905de73e234b644d8ae8886d067f1 (99ms) I0512 03:02:27.888238 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (101ms) I0512 03:02:27.988947 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0512 03:02:28.088285 1 gather_workloads_info.go:387] No image sha256:fac526ab9335a9a8decb028a30673f4ae718deae8da114f36103b6438ba6458f (99ms) I0512 03:02:28.188942 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0512 03:02:28.288130 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (99ms) I0512 03:02:28.388369 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (100ms) I0512 03:02:28.388398 1 tasks_processing.go:74] worker 1 stopped. I0512 03:02:28.388652 1 recorder.go:75] Recording config/workload_info with fingerprint=748205fadf92e6c4778ceaef65e33282cec728a8cb78139cd8aa5304e6403658 I0512 03:02:28.388666 1 gather.go:177] gatherer "workloads" function "workload_info" took 6.940018307s to process 1 records I0512 03:02:28.388679 1 periodic.go:256] Periodic gather workloads completed in 6.94s I0512 03:02:28.388687 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0512 03:02:28.388692 1 periodic.go:209] Running conditional gatherer I0512 03:02:28.395289 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0512 03:02:28.582649 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0512 03:02:28.608988 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: no such host I0512 03:02:28.609065 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0512 03:02:28.616043 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0512 03:02:28.616056 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616063 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616066 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616069 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616072 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616074 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616078 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616081 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616083 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0512 03:02:28.616087 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0512 03:02:28.616100 1 tasks_processing.go:45] number of workers: 3 I0512 03:02:28.616107 1 tasks_processing.go:69] worker 2 listening for tasks. I0512 03:02:28.616111 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0512 03:02:28.616116 1 tasks_processing.go:69] worker 0 listening for tasks. I0512 03:02:28.616128 1 tasks_processing.go:69] worker 1 listening for tasks. I0512 03:02:28.616136 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0512 03:02:28.616140 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0512 03:02:28.616151 1 tasks_processing.go:74] worker 0 stopped. I0512 03:02:28.616189 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0512 03:02:28.616203 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.184µs to process 1 records I0512 03:02:28.616266 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=9fafdcbf373f1f367f1d69551ea59114592cce1681189be0f3dde774e0530661 I0512 03:02:28.616274 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.363µs to process 1 records I0512 03:02:28.616285 1 tasks_processing.go:74] worker 1 stopped. I0512 03:02:28.616328 1 tasks_processing.go:74] worker 2 stopped. I0512 03:02:28.616339 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 212.783µs to process 0 records I0512 03:02:28.616362 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0512 03:02:28.616379 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0512 03:02:28.641200 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=5d0b8e461ca3fcca79e769e3d59b77427c3c24769d687a74da479d283c35921c I0512 03:02:28.641318 1 diskrecorder.go:70] Writing 121 records to /var/lib/insights-operator/insights-2026-05-12-030228.tar.gz I0512 03:02:28.649087 1 diskrecorder.go:51] Wrote 121 records to disk in 7ms I0512 03:02:28.649113 1 periodic.go:278] Gathering cluster info every 2h0m0s I0512 03:02:28.649126 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0512 03:03:37.635709 1 diskrecorder.go:223] Found files to send: insights-2026-05-12-030228.tar.gz I0512 03:03:37.635738 1 insightsuploader.go:150] Checking archives to upload periodically every 15m46.431938315s I0512 03:03:37.635746 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0512 03:03:37.648025 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0512 03:03:37.908000 1 requests.go:87] Successfully reported id=2026-05-12T03:03:37Z x-rh-insights-request-id=598f3c03d38641a7b6c4817a1248928d, wrote=61424 I0512 03:03:37.908063 1 insightsuploader.go:187] Uploaded report successfully in 272.306492ms I0512 03:03:37.908095 1 controller.go:128] Initializing last reported time to 2026-05-12T03:03:37Z I0512 03:03:37.908161 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0512 03:03:37.908175 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0512 03:03:37.908184 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0512 03:03:37.914991 1 controller.go:512] The operator is healthy I0512 03:04:07.664944 1 controller.go:512] The operator is healthy I0512 03:04:42.573965 1 insightsreport.go:137] Pulling report from smart-proxy I0512 03:04:42.573989 1 insightsreport.go:149] Retrieving report I0512 03:04:42.581585 1 requests.go:110] Retrieving report for cluster: 68873eb4-3a55-47cc-8a47-664a767e9907 I0512 03:04:42.581597 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/68873eb4-3a55-47cc-8a47-664a767e9907/reports I0512 03:04:42.584336 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/68873eb4-3a55-47cc-8a47-664a767e9907/reports I0512 03:04:42.758098 1 insightsreport.go:184] Report retrieved I0512 03:04:42.783955 1 insightsreport.go:239] Report retrieved correctly I0512 03:06:07.665156 1 controller.go:512] The operator is healthy I0512 03:07:07.628366 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0512 03:07:07.634599 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:07:07.634616 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0512 03:07:07.639247 1 secretconfigobserver.go:119] support secret does not exist I0512 03:08:07.664191 1 controller.go:512] The operator is healthy I0512 03:10:07.665035 1 controller.go:512] The operator is healthy I0512 03:12:07.626491 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 03:12:07.639411 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0512 03:12:07.639486 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 03:12:07.654773 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:12:07.654791 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0512 03:12:07.662842 1 secretconfigobserver.go:119] support secret does not exist I0512 03:12:07.667234 1 controller.go:512] The operator is healthy I0512 03:14:07.666770 1 controller.go:512] The operator is healthy I0512 03:16:07.665135 1 controller.go:512] The operator is healthy I0512 03:17:07.663770 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0512 03:17:07.670221 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:17:07.670240 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0512 03:17:07.674880 1 secretconfigobserver.go:119] support secret does not exist I0512 03:18:07.665199 1 controller.go:512] The operator is healthy I0512 03:19:29.341101 1 insightsuploader.go:146] Nothing to report since 2026-05-12T03:03:37Z I0512 03:20:07.664489 1 controller.go:512] The operator is healthy I0512 03:22:07.624758 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 03:22:07.663690 1 controller.go:512] The operator is healthy I0512 03:22:07.675676 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0512 03:22:07.680598 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:22:07.680613 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0512 03:22:07.685185 1 secretconfigobserver.go:119] support secret does not exist I0512 03:24:07.664873 1 controller.go:512] The operator is healthy I0512 03:26:07.664939 1 controller.go:512] The operator is healthy I0512 03:27:07.685936 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0512 03:27:07.691400 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:27:07.691418 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0512 03:27:07.697858 1 secretconfigobserver.go:119] support secret does not exist I0512 03:28:07.665368 1 controller.go:512] The operator is healthy I0512 03:30:07.665966 1 controller.go:512] The operator is healthy I0512 03:32:07.625178 1 configmapobserver.go:84] configmaps "insights-config" not found I0512 03:32:07.664405 1 controller.go:512] The operator is healthy I0512 03:32:07.698604 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0512 03:32:07.703931 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0512 03:32:07.703948 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0512 03:32:07.709817 1 secretconfigobserver.go:119] support secret does not exist I0512 03:34:07.663914 1 controller.go:512] The operator is healthy