I0506 12:47:48.369128 1 cmd.go:253] Using service-serving-cert provided certificates I0506 12:47:48.369574 1 observer_polling.go:159] Starting file observer I0506 12:47:48.578904 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0506 12:47:48.579087 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0506 12:47:48.579775 1 secure_serving.go:57] Forcing use of http/1.1 only W0506 12:47:48.579796 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0506 12:47:48.579801 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. I0506 12:47:48.579799 1 simple_featuregate_reader.go:171] Starting feature-gate-detector W0506 12:47:48.579805 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0506 12:47:48.579824 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0506 12:47:48.579826 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0506 12:47:48.579829 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0506 12:47:48.583403 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0506 12:47:48.583428 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"60aa0293-389b-46f2-9f5f-a85899d6c726", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0506 12:47:48.584833 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0506 12:47:48.584852 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0506 12:47:48.584852 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0506 12:47:48.584861 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0506 12:47:48.584876 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0506 12:47:48.584867 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0506 12:47:48.585187 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0506 12:47:48.585442 1 secure_serving.go:213] Serving securely on [::]:8443 I0506 12:47:48.585470 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0506 12:47:48.588713 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0506 12:47:48.588730 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0506 12:47:48.588816 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0506 12:47:48.597923 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 12:47:48.597939 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0506 12:47:48.602646 1 secretconfigobserver.go:119] support secret does not exist I0506 12:47:48.607260 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 12:47:48.611813 1 secretconfigobserver.go:119] support secret does not exist I0506 12:47:48.614747 1 recorder.go:176] Pruning old reports every 6h40m41s, max age is 288h0m0s I0506 12:47:48.619476 1 periodic.go:216] Running clusterconfig gatherer I0506 12:47:48.619482 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0506 12:47:48.619499 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0506 12:47:48.619509 1 tasks_processing.go:45] number of workers: 64 I0506 12:47:48.619518 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0506 12:47:48.619524 1 insightsreport.go:296] Starting report retriever I0506 12:47:48.619530 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0506 12:47:48.619541 1 tasks_processing.go:69] worker 29 listening for tasks. I0506 12:47:48.619551 1 tasks_processing.go:69] worker 20 listening for tasks. I0506 12:47:48.619546 1 tasks_processing.go:69] worker 19 listening for tasks. I0506 12:47:48.619563 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 12:47:48.619571 1 tasks_processing.go:71] worker 19 working on networks task. I0506 12:47:48.619572 1 tasks_processing.go:69] worker 26 listening for tasks. I0506 12:47:48.619571 1 tasks_processing.go:69] worker 30 listening for tasks. I0506 12:47:48.619567 1 tasks_processing.go:69] worker 9 listening for tasks. I0506 12:47:48.619583 1 tasks_processing.go:69] worker 31 listening for tasks. I0506 12:47:48.619587 1 tasks_processing.go:71] worker 9 working on authentication task. I0506 12:47:48.619594 1 tasks_processing.go:69] worker 10 listening for tasks. I0506 12:47:48.619597 1 tasks_processing.go:69] worker 32 listening for tasks. I0506 12:47:48.619579 1 tasks_processing.go:71] worker 0 working on openstack_dataplanenodesets task. I0506 12:47:48.619624 1 tasks_processing.go:69] worker 33 listening for tasks. I0506 12:47:48.619623 1 tasks_processing.go:69] worker 24 listening for tasks. I0506 12:47:48.619638 1 tasks_processing.go:69] worker 21 listening for tasks. I0506 12:47:48.619642 1 tasks_processing.go:69] worker 34 listening for tasks. I0506 12:47:48.619649 1 tasks_processing.go:69] worker 22 listening for tasks. I0506 12:47:48.619651 1 tasks_processing.go:69] worker 35 listening for tasks. I0506 12:47:48.619650 1 tasks_processing.go:69] worker 15 listening for tasks. I0506 12:47:48.619657 1 tasks_processing.go:69] worker 23 listening for tasks. I0506 12:47:48.619560 1 tasks_processing.go:69] worker 42 listening for tasks. I0506 12:47:48.619666 1 tasks_processing.go:69] worker 13 listening for tasks. I0506 12:47:48.619658 1 tasks_processing.go:69] worker 36 listening for tasks. I0506 12:47:48.619661 1 tasks_processing.go:69] worker 7 listening for tasks. I0506 12:47:48.619677 1 tasks_processing.go:69] worker 38 listening for tasks. I0506 12:47:48.619669 1 tasks_processing.go:69] worker 54 listening for tasks. I0506 12:47:48.619681 1 tasks_processing.go:69] worker 28 listening for tasks. I0506 12:47:48.619677 1 tasks_processing.go:69] worker 12 listening for tasks. I0506 12:47:48.619691 1 tasks_processing.go:71] worker 22 working on support_secret task. I0506 12:47:48.619693 1 tasks_processing.go:69] worker 40 listening for tasks. I0506 12:47:48.619651 1 tasks_processing.go:69] worker 8 listening for tasks. I0506 12:47:48.619692 1 tasks_processing.go:69] worker 27 listening for tasks. I0506 12:47:48.619702 1 tasks_processing.go:71] worker 29 working on machine_config_pools task. I0506 12:47:48.619705 1 tasks_processing.go:71] worker 42 working on tsdb_status task. I0506 12:47:48.619708 1 tasks_processing.go:71] worker 27 working on clusterroles task. I0506 12:47:48.619708 1 tasks_processing.go:69] worker 17 listening for tasks. I0506 12:47:48.619713 1 tasks_processing.go:69] worker 59 listening for tasks. I0506 12:47:48.619718 1 tasks_processing.go:69] worker 2 listening for tasks. I0506 12:47:48.619726 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 12:47:48.619577 1 tasks_processing.go:69] worker 5 listening for tasks. I0506 12:47:48.619731 1 tasks_processing.go:69] worker 61 listening for tasks. I0506 12:47:48.619735 1 tasks_processing.go:69] worker 62 listening for tasks. I0506 12:47:48.619738 1 tasks_processing.go:71] worker 32 working on operators_pods_and_events task. I0506 12:47:48.619742 1 tasks_processing.go:71] worker 31 working on aggregated_monitoring_cr_names task. I0506 12:47:48.619746 1 tasks_processing.go:69] worker 52 listening for tasks. I0506 12:47:48.619701 1 tasks_processing.go:69] worker 46 listening for tasks. I0506 12:47:48.619751 1 tasks_processing.go:69] worker 53 listening for tasks. I0506 12:47:48.619725 1 tasks_processing.go:69] worker 49 listening for tasks. I0506 12:47:48.619585 1 tasks_processing.go:71] worker 30 working on dvo_metrics task. I0506 12:47:48.619779 1 tasks_processing.go:69] worker 57 listening for tasks. I0506 12:47:48.619791 1 tasks_processing.go:71] worker 17 working on ingress_certificates task. I0506 12:47:48.619794 1 tasks_processing.go:71] worker 62 working on feature_gates task. I0506 12:47:48.619797 1 tasks_processing.go:71] worker 57 working on pod_network_connectivity_checks task. I0506 12:47:48.619782 1 tasks_processing.go:71] worker 49 working on openstack_version task. I0506 12:47:48.619809 1 tasks_processing.go:71] worker 2 working on cluster_apiserver task. I0506 12:47:48.619812 1 tasks_processing.go:71] worker 59 working on ceph_cluster task. I0506 12:47:48.619692 1 tasks_processing.go:71] worker 35 working on container_runtime_configs task. I0506 12:47:48.619823 1 tasks_processing.go:71] worker 46 working on machine_sets task. I0506 12:47:48.619834 1 tasks_processing.go:71] worker 1 working on container_images task. I0506 12:47:48.619837 1 tasks_processing.go:71] worker 53 working on schedulers task. I0506 12:47:48.619845 1 tasks_processing.go:71] worker 52 working on sap_config task. I0506 12:47:48.619737 1 tasks_processing.go:69] worker 50 listening for tasks. I0506 12:47:48.619855 1 tasks_processing.go:71] worker 50 working on qemu_kubevirt_launcher_logs task. I0506 12:47:48.619705 1 tasks_processing.go:71] worker 8 working on storage_cluster task. I0506 12:47:48.619675 1 tasks_processing.go:69] worker 43 listening for tasks. I0506 12:47:48.619679 1 tasks_processing.go:69] worker 16 listening for tasks. I0506 12:47:48.619681 1 tasks_processing.go:69] worker 44 listening for tasks. I0506 12:47:48.619686 1 tasks_processing.go:69] worker 39 listening for tasks. I0506 12:47:48.619685 1 tasks_processing.go:69] worker 60 listening for tasks. I0506 12:47:48.619686 1 tasks_processing.go:71] worker 38 working on pdbs task. I0506 12:47:48.619690 1 tasks_processing.go:71] worker 54 working on olm_operators task. I0506 12:47:48.619905 1 tasks_processing.go:71] worker 60 working on mutating_webhook_configurations task. I0506 12:47:48.619906 1 tasks_processing.go:71] worker 16 working on service_accounts task. I0506 12:47:48.619908 1 tasks_processing.go:71] worker 39 working on openstack_controlplanes task. I0506 12:47:48.619906 1 tasks_processing.go:71] worker 44 working on nodenetworkconfigurationpolicies task. I0506 12:47:48.619744 1 tasks_processing.go:69] worker 51 listening for tasks. I0506 12:47:48.619971 1 tasks_processing.go:71] worker 51 working on machines task. I0506 12:47:48.619692 1 tasks_processing.go:69] worker 45 listening for tasks. I0506 12:47:48.619694 1 tasks_processing.go:69] worker 55 listening for tasks. I0506 12:47:48.619696 1 tasks_processing.go:71] worker 28 working on machine_healthchecks task. I0506 12:47:48.620016 1 tasks_processing.go:71] worker 55 working on image_pruners task. I0506 12:47:48.619697 1 tasks_processing.go:71] worker 15 working on machine_configs task. I0506 12:47:48.620033 1 tasks_processing.go:71] worker 45 working on openshift_logging task. I0506 12:47:48.619667 1 tasks_processing.go:69] worker 37 listening for tasks. I0506 12:47:48.620065 1 tasks_processing.go:71] worker 37 working on lokistack task. I0506 12:47:48.619697 1 tasks_processing.go:71] worker 12 working on openstack_dataplanedeployments task. I0506 12:47:48.619701 1 tasks_processing.go:71] worker 23 working on infrastructures task. I0506 12:47:48.619703 1 tasks_processing.go:69] worker 58 listening for tasks. I0506 12:47:48.619661 1 tasks_processing.go:69] worker 18 listening for tasks. I0506 12:47:48.619706 1 tasks_processing.go:69] worker 41 listening for tasks. I0506 12:47:48.620274 1 tasks_processing.go:71] worker 58 working on oauths task. I0506 12:47:48.620345 1 tasks_processing.go:71] worker 41 working on version task. I0506 12:47:48.620400 1 tasks_processing.go:71] worker 18 working on validating_webhook_configurations task. I0506 12:47:48.619692 1 tasks_processing.go:69] worker 11 listening for tasks. I0506 12:47:48.619710 1 tasks_processing.go:69] worker 47 listening for tasks. I0506 12:47:48.619711 1 tasks_processing.go:71] worker 40 working on node_logs task. I0506 12:47:48.620477 1 tasks_processing.go:71] worker 11 working on nodenetworkstates task. I0506 12:47:48.619717 1 tasks_processing.go:69] worker 48 listening for tasks. I0506 12:47:48.619714 1 tasks_processing.go:71] worker 26 working on openshift_machine_api_events task. I0506 12:47:48.620674 1 tasks_processing.go:71] worker 48 working on image_registries task. I0506 12:47:48.620790 1 tasks_processing.go:71] worker 47 working on operators task. I0506 12:47:48.619714 1 tasks_processing.go:69] worker 63 listening for tasks. I0506 12:47:48.620957 1 tasks_processing.go:71] worker 63 working on install_plans task. I0506 12:47:48.619706 1 tasks_processing.go:69] worker 4 listening for tasks. I0506 12:47:48.619723 1 tasks_processing.go:71] worker 36 working on sap_datahubs task. I0506 12:47:48.619723 1 tasks_processing.go:71] worker 7 working on monitoring_persistent_volumes task. I0506 12:47:48.621195 1 tasks_processing.go:71] worker 4 working on sap_pods task. I0506 12:47:48.619730 1 tasks_processing.go:71] worker 13 working on active_alerts task. I0506 12:47:48.619734 1 tasks_processing.go:71] worker 33 working on proxies task. I0506 12:47:48.619730 1 tasks_processing.go:69] worker 56 listening for tasks. I0506 12:47:48.622404 1 tasks_processing.go:71] worker 56 working on metrics task. I0506 12:47:48.619731 1 tasks_processing.go:69] worker 3 listening for tasks. I0506 12:47:48.622504 1 tasks_processing.go:71] worker 3 working on overlapping_namespace_uids task. I0506 12:47:48.619737 1 tasks_processing.go:71] worker 10 working on nodes task. I0506 12:47:48.619743 1 tasks_processing.go:71] worker 21 working on cost_management_metrics_configs task. I0506 12:47:48.619744 1 tasks_processing.go:71] worker 34 working on silenced_alerts task. I0506 12:47:48.619741 1 tasks_processing.go:71] worker 20 working on machine_autoscalers task. I0506 12:47:48.619744 1 tasks_processing.go:71] worker 24 working on certificate_signing_requests task. I0506 12:47:48.619565 1 tasks_processing.go:69] worker 25 listening for tasks. I0506 12:47:48.619787 1 tasks_processing.go:71] worker 5 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0506 12:47:48.619790 1 tasks_processing.go:71] worker 61 working on crds task. I0506 12:47:48.619673 1 tasks_processing.go:69] worker 14 listening for tasks. I0506 12:47:48.619670 1 tasks_processing.go:69] worker 6 listening for tasks. I0506 12:47:48.619908 1 tasks_processing.go:71] worker 43 working on jaegers task. I0506 12:47:48.624436 1 tasks_processing.go:71] worker 25 working on node_features task. I0506 12:47:48.624738 1 tasks_processing.go:71] worker 14 working on image task. I0506 12:47:48.624751 1 tasks_processing.go:71] worker 0 working on storage_classes task. I0506 12:47:48.624770 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 5.129909ms to process 0 records I0506 12:47:48.624779 1 tasks_processing.go:71] worker 6 working on ingress task. E0506 12:47:48.627241 1 gather_silenced_alerts.go:52] Unable to retrieve silenced alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48671->172.30.0.10:53: read: connection refused E0506 12:47:48.627248 1 gather_active_alerts.go:64] Unable to retrieve most recent alerts: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48671->172.30.0.10:53: read: connection refused I0506 12:47:48.627258 1 tasks_processing.go:71] worker 34 working on config_maps task. E0506 12:47:48.627265 1 gather.go:140] gatherer "clusterconfig" function "silenced_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=false&inhibited=false&silenced=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48671->172.30.0.10:53: read: connection refused I0506 12:47:48.627275 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 3.936401ms to process 0 records E0506 12:47:48.627267 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48578->172.30.0.10:53: read: connection refused E0506 12:47:48.627285 1 gather.go:140] gatherer "clusterconfig" function "active_alerts" failed with the error: Get "https://alertmanager-main.openshift-monitoring.svc:9094/api/v2/alerts?active=true": dial tcp: lookup alertmanager-main.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48671->172.30.0.10:53: read: connection refused I0506 12:47:48.627295 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 5.766568ms to process 0 records E0506 12:47:48.627303 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48578->172.30.0.10:53: read: connection refused E0506 12:47:48.627304 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48578->172.30.0.10:53: read: connection refused I0506 12:47:48.627316 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 4.843603ms to process 0 records E0506 12:47:48.627327 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:48578->172.30.0.10:53: read: connection refused I0506 12:47:48.627340 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 7.603205ms to process 0 records I0506 12:47:48.627359 1 tasks_processing.go:74] worker 13 stopped. I0506 12:47:48.627368 1 tasks_processing.go:74] worker 56 stopped. I0506 12:47:48.627371 1 tasks_processing.go:74] worker 42 stopped. I0506 12:47:48.628324 1 tasks_processing.go:74] worker 22 stopped. E0506 12:47:48.628338 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0506 12:47:48.628349 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 8.625332ms to process 0 records I0506 12:47:48.628612 1 tasks_processing.go:74] worker 19 stopped. I0506 12:47:48.628786 1 recorder.go:75] Recording config/network with fingerprint=e952e98b5e1193e87a91f52ec5d84c9d8f6b98b6c81535011b2778357e6de99b I0506 12:47:48.628799 1 gather.go:177] gatherer "clusterconfig" function "networks" took 9.027867ms to process 1 records I0506 12:47:48.642383 1 tasks_processing.go:74] worker 12 stopped. I0506 12:47:48.642395 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 22.261509ms to process 0 records I0506 12:47:48.642406 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 22.342103ms to process 0 records I0506 12:47:48.642412 1 tasks_processing.go:74] worker 45 stopped. I0506 12:47:48.642504 1 tasks_processing.go:74] worker 29 stopped. I0506 12:47:48.642512 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 22.794702ms to process 0 records I0506 12:47:48.642584 1 tasks_processing.go:74] worker 52 stopped. I0506 12:47:48.642592 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 22.733465ms to process 0 records I0506 12:47:48.642597 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 22.517543ms to process 0 records I0506 12:47:48.642601 1 tasks_processing.go:74] worker 37 stopped. I0506 12:47:48.642654 1 tasks_processing.go:74] worker 4 stopped. I0506 12:47:48.642669 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 21.446415ms to process 0 records I0506 12:47:48.642678 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 21.68569ms to process 0 records I0506 12:47:48.642686 1 tasks_processing.go:74] worker 36 stopped. I0506 12:47:48.642702 1 tasks_processing.go:74] worker 49 stopped. I0506 12:47:48.642718 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 22.887591ms to process 0 records I0506 12:47:48.642731 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 19.688995ms to process 0 records I0506 12:47:48.642737 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 22.890956ms to process 0 records I0506 12:47:48.642742 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 22.762154ms to process 0 records I0506 12:47:48.642748 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 22.230456ms to process 0 records I0506 12:47:48.642748 1 tasks_processing.go:74] worker 21 stopped. I0506 12:47:48.642757 1 tasks_processing.go:74] worker 59 stopped. I0506 12:47:48.642759 1 tasks_processing.go:74] worker 44 stopped. I0506 12:47:48.642758 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 22.79923ms to process 0 records I0506 12:47:48.642763 1 tasks_processing.go:74] worker 39 stopped. I0506 12:47:48.642777 1 tasks_processing.go:74] worker 11 stopped. I0506 12:47:48.642815 1 tasks_processing.go:74] worker 9 stopped. I0506 12:47:48.642833 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0506 12:47:48.642846 1 controller.go:368] The initial operator extension status is healthy I0506 12:47:48.642854 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0506 12:47:48.642859 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0506 12:47:48.642862 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0506 12:47:48.642878 1 controller.go:531] The operator is still being initialized I0506 12:47:48.642883 1 controller.go:554] The operator is healthy I0506 12:47:48.642999 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0506 12:47:48.643006 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0506 12:47:48.643028 1 recorder.go:75] Recording config/authentication with fingerprint=25fc58e009df2fe06b11d59753753b5e22659a3a8bd4518a11ffe87d445df104 I0506 12:47:48.643044 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 23.213952ms to process 1 records W0506 12:47:48.643056 1 operator.go:328] started I0506 12:47:48.643072 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0506 12:47:48.643110 1 recorder.go:75] Recording config/proxy with fingerprint=35b271a97034a9735a391b6531ba32d7f739bbb6b7552ec97827f595242b206e I0506 12:47:48.643122 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 20.794013ms to process 1 records E0506 12:47:48.643131 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0506 12:47:48.643140 1 gather.go:177] gatherer "clusterconfig" function "machines" took 22.939248ms to process 0 records I0506 12:47:48.643150 1 tasks_processing.go:74] worker 33 stopped. I0506 12:47:48.643157 1 tasks_processing.go:74] worker 51 stopped. I0506 12:47:48.643187 1 tasks_processing.go:74] worker 58 stopped. I0506 12:47:48.643343 1 recorder.go:75] Recording config/oauth with fingerprint=7eb191cfc2d85dd40bff737946e61244723736768824bb4ab51c95e148558dc0 I0506 12:47:48.643355 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 22.812054ms to process 1 records I0506 12:47:48.652631 1 tasks_processing.go:74] worker 26 stopped. I0506 12:47:48.652648 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 32.014614ms to process 0 records I0506 12:47:48.652756 1 tasks_processing.go:74] worker 7 stopped. I0506 12:47:48.652769 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 31.658986ms to process 0 records I0506 12:47:48.653013 1 tasks_processing.go:74] worker 2 stopped. I0506 12:47:48.653163 1 recorder.go:75] Recording config/apiserver with fingerprint=132aaf825a4f03fed3391f107cbb3244e57d55cfe4de41d326edb5985d733bb4 I0506 12:47:48.653197 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 33.195114ms to process 1 records I0506 12:47:48.653281 1 tasks_processing.go:74] worker 55 stopped. I0506 12:47:48.653417 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=0c0d59323aa5b8a76afd6cfc68a846ebbe99953666debedeab861a693822ef12 I0506 12:47:48.653428 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 33.096984ms to process 1 records I0506 12:47:48.653521 1 tasks_processing.go:74] worker 23 stopped. I0506 12:47:48.654662 1 recorder.go:75] Recording config/infrastructure with fingerprint=dc6043ac7b34065a016a75a01a722ddd7a699659fa6c2070ecd51a83e0436e60 I0506 12:47:48.654730 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 33.028012ms to process 1 records I0506 12:47:48.654764 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 34.116833ms to process 0 records I0506 12:47:48.654806 1 tasks_processing.go:74] worker 40 stopped. I0506 12:47:48.654860 1 tasks_processing.go:74] worker 62 stopped. I0506 12:47:48.655299 1 recorder.go:75] Recording config/featuregate with fingerprint=fc8f2cef876e5f7421885dce82b15b6b1bb3a54900b90288e4568d50c471c6c0 I0506 12:47:48.655321 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 35.054167ms to process 1 records I0506 12:47:48.655754 1 tasks_processing.go:74] worker 8 stopped. I0506 12:47:48.655778 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 35.886648ms to process 0 records I0506 12:47:48.655999 1 tasks_processing.go:74] worker 53 stopped. I0506 12:47:48.656137 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=931ff3c0a4bca6a16f36256391c265272fb2b6a5741aa9860d3b0e4f8cfd38e9 I0506 12:47:48.656206 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 36.148177ms to process 1 records I0506 12:47:48.656901 1 tasks_processing.go:74] worker 48 stopped. I0506 12:47:48.657352 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=5993ef0501e00022470007c71e99ac36a6ac79ffa888c5614ea0d855d9a95011 I0506 12:47:48.657372 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 36.214326ms to process 1 records W0506 12:47:48.658008 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 12:47:48.658366 1 tasks_processing.go:74] worker 35 stopped. I0506 12:47:48.658377 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 38.53943ms to process 0 records I0506 12:47:48.658383 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 38.54129ms to process 0 records I0506 12:47:48.658388 1 tasks_processing.go:74] worker 46 stopped. I0506 12:47:48.669333 1 tasks_processing.go:74] worker 20 stopped. I0506 12:47:48.669348 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 45.978721ms to process 0 records I0506 12:47:48.669968 1 tasks_processing.go:74] worker 15 stopped. E0506 12:47:48.669983 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0506 12:47:48.670021 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0506 12:47:48.670032 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 49.934199ms to process 1 records I0506 12:47:48.670112 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=2358351374f3fdd84f8aa65f6dbafd7bc25e443240e8847ba9c14fc53c27848e I0506 12:47:48.670127 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=f650b186fc241dbe367ba563638a998e3eefd933dbec35afe2cc8d4681ce71c9 I0506 12:47:48.670133 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 45.228914ms to process 2 records I0506 12:47:48.670135 1 gather_logs.go:145] no pods in namespace were found I0506 12:47:48.670142 1 tasks_processing.go:74] worker 0 stopped. I0506 12:47:48.670148 1 tasks_processing.go:74] worker 50 stopped. I0506 12:47:48.670155 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 50.284294ms to process 0 records I0506 12:47:48.670200 1 tasks_processing.go:74] worker 10 stopped. I0506 12:47:48.670602 1 recorder.go:75] Recording config/node/ip-10-0-0-131.ec2.internal with fingerprint=aaf9ec45ef60fa087176d5baa605cb56f0b197658541eae90c8e89e76eb052fc I0506 12:47:48.670684 1 recorder.go:75] Recording config/node/ip-10-0-1-148.ec2.internal with fingerprint=a5095b1828ebffaa4543492d5adba188578d9ac17ccecc32111e5ebd42440908 I0506 12:47:48.670765 1 recorder.go:75] Recording config/node/ip-10-0-2-30.ec2.internal with fingerprint=bbc04a3ff4194e7f086411441c64fbacb01bd8cdb1d8190ca09e9563e409f292 I0506 12:47:48.670774 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 47.566524ms to process 3 records I0506 12:47:48.670889 1 tasks_processing.go:74] worker 43 stopped. I0506 12:47:48.670902 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 47.484079ms to process 0 records E0506 12:47:48.673502 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0506 12:47:48.673515 1 tasks_processing.go:74] worker 57 stopped. E0506 12:47:48.673522 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0506 12:47:48.673533 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 53.707139ms to process 0 records I0506 12:47:48.673546 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 49.07282ms to process 0 records I0506 12:47:48.673553 1 tasks_processing.go:74] worker 25 stopped. I0506 12:47:48.673563 1 tasks_processing.go:74] worker 28 stopped. E0506 12:47:48.673574 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0506 12:47:48.673588 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 53.56889ms to process 0 records I0506 12:47:48.673734 1 tasks_processing.go:74] worker 14 stopped. I0506 12:47:48.673842 1 recorder.go:75] Recording config/image with fingerprint=b2830bb125cefa78d33b3eed0d25e1d1451b4a77eef33ba676223a79aef79527 I0506 12:47:48.673859 1 gather.go:177] gatherer "clusterconfig" function "image" took 48.98041ms to process 1 records I0506 12:47:48.673973 1 tasks_processing.go:74] worker 18 stopped. I0506 12:47:48.674029 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=87c974b6a8c9139d05b04d8a25ee64d7380a2d172ba1c706f61bc73cf9e875c8 I0506 12:47:48.674100 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=98d2e4c12870e86c4ef2a93decde2e9e8825677200c99027fbe1cec2ebf5770e I0506 12:47:48.674255 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=68f318ebfd0c923b197c6f147c497b0d9245f7296b5179f10054d3876694cf4b I0506 12:47:48.674321 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=095e3ec8261f10a6798b3d3cfd588387d52043ce37491b461d6674fedec02e76 I0506 12:47:48.674351 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=925e803116e4ad943c5214c0ba10e8b6a57c33332c9d9220000e541ad70a7656 I0506 12:47:48.674378 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=11c4cb4b8bbd4f6013d57f70fb65eebd8633b6f17b65b21ee51d56b031f7fdce I0506 12:47:48.674408 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=06316f170cba6d3438f488a9c7b0b092a0428274a880cad82cdb5a7e8fef182d I0506 12:47:48.674940 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=dec848e260fd06971b2d58f764cad7d014a0cbe0bbdb9e269bd8115dc8cb760e I0506 12:47:48.675457 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=3111adb2a359b26b1903d2da4ba366b75ce681c309debb758f080694720be0ea I0506 12:47:48.675744 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=fab4ed83b7d15b3eb681db32f9a9bea90173a7e442270e130ab026a9a55696b7 I0506 12:47:48.675845 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=d39a19bbcbb3661f7c3c305a0445a3cb1bdc17d6d82e5b7425ca55b0684c6779 I0506 12:47:48.675961 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=77bc4c8f3f281de880afbbbf343f14b8a371c8b1ada040b615ce486e81aaac8e I0506 12:47:48.676086 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=3490ef82e8c1e0399e0b872d13a5f5708aa8e31a508396ce49eedf05ba6c94df I0506 12:47:48.676125 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 53.377914ms to process 13 records I0506 12:47:48.676408 1 tasks_processing.go:74] worker 6 stopped. I0506 12:47:48.676639 1 recorder.go:75] Recording config/ingress with fingerprint=8de85bf4c4669c13fd4791a3fe1b2c980c5c46bc23d8df2b6fb9213329a318f3 I0506 12:47:48.676674 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 49.152449ms to process 1 records I0506 12:47:48.676931 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=3396cead1c75853f4494ce56d959f72d12b64923aaa00255722a122d0c4a1aa8 I0506 12:47:48.677051 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=8f1234471e31b65a61180fe7b4aa73347e243c4088adc08f4afb96856ca5bf2d I0506 12:47:48.677074 1 tasks_processing.go:74] worker 60 stopped. I0506 12:47:48.677191 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=d758475083d725e4959ead59aa54b449ec3ca04a4cd145b689241e3860f74a9b I0506 12:47:48.677208 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 54.291793ms to process 3 records I0506 12:47:48.677330 1 tasks_processing.go:74] worker 54 stopped. I0506 12:47:48.677444 1 recorder.go:75] Recording config/olm_operators with fingerprint=98f5199fd2df4309176ea9388d652acab5613ecb2f867cf37858b314f84455d7 I0506 12:47:48.677501 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 54.468367ms to process 1 records I0506 12:47:48.677851 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0506 12:47:48.677867 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 54.684617ms to process 1 records I0506 12:47:48.677874 1 tasks_processing.go:74] worker 3 stopped. I0506 12:47:48.678627 1 tasks_processing.go:74] worker 38 stopped. I0506 12:47:48.678767 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=cc474a0e3aee64bc1a42ea5681eb267960689dab3f485215b631287adbd1eaf8 I0506 12:47:48.678801 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=1cac99002b957f24e4934fd6264b685f9187b8969e58eab50386fcd465fa7be2 I0506 12:47:48.678825 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=1fb896e4f6b39b9c62c4d3d0b03db8eef30360a95dd91b49a73450414628e1a9 I0506 12:47:48.678848 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=af66eb7091083ba0a6187aa419a8d1f4a57f80ff9d5cc530cf9b60ae413a26be I0506 12:47:48.678872 1 controller.go:254] Source scaController *sca.Controller is not ready I0506 12:47:48.678891 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0506 12:47:48.678896 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0506 12:47:48.678900 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0506 12:47:48.678904 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0506 12:47:48.678874 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=be68294fba2dddc8da313c7301d4c6e2f49531607f188b3b6403df9c9c0d4078 I0506 12:47:48.678941 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=08bbd6977809c84b16b69f6355523d14ab24cc9dd9f51940fb328b9050ba96ae I0506 12:47:48.678954 1 controller.go:531] The operator is still being initialized I0506 12:47:48.678965 1 controller.go:554] The operator is healthy I0506 12:47:48.678974 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=876b0086f24b229af08c4917b7b8d952814f324cd7eb3137eebe38bb31afb1a2 I0506 12:47:48.678996 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=9cde31b2d07d07830115387c02e62d60105957905d02ccab94e264aa3fd894ac I0506 12:47:48.679014 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=0dcfe8ea993509fb6df8287df373ba3056a71df7c3511f45bbcc0c3573942831 I0506 12:47:48.679036 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=c7803bcba3afd9ee7717923d1540d7eca2b7f9b4e87897b76943d627411d9125 I0506 12:47:48.679057 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=33875075fdd6b4e09935882d21099b429014cfa4838cef3dddee035286ffbda5 I0506 12:47:48.679065 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 58.733742ms to process 11 records I0506 12:47:48.679157 1 tasks_processing.go:74] worker 27 stopped. I0506 12:47:48.679304 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=5df2a13908ff4f41f49fbe580a12e4c0923deedf336b95addb179ffe1c48b832 I0506 12:47:48.679459 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=0f24572bed653beb827d89c51223678391d026ff79d340ca2de534afbbe6db69 I0506 12:47:48.679472 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 59.028696ms to process 2 records I0506 12:47:48.681260 1 tasks_processing.go:74] worker 24 stopped. I0506 12:47:48.681277 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 57.887669ms to process 0 records I0506 12:47:48.685341 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0506 12:47:48.685367 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0506 12:47:48.685381 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0506 12:47:48.688892 1 base_controller.go:82] Caches are synced for ConfigController I0506 12:47:48.688906 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... E0506 12:47:48.691938 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%276309b07b-3e1f-48b4-ad06-4144fecb5ba4%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:43508->172.30.0.10:53: read: connection refused I0506 12:47:48.691950 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%276309b07b-3e1f-48b4-ad06-4144fecb5ba4%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:43508->172.30.0.10:53: read: connection refused I0506 12:47:48.711770 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 12:47:48.715378 1 tasks_processing.go:74] worker 41 stopped. I0506 12:47:48.715736 1 recorder.go:75] Recording config/version with fingerprint=4fbd29b96c9d085ddac15e5aa972f8c79d363545ddea28a25e6d98b7b1eee001 I0506 12:47:48.715758 1 recorder.go:75] Recording config/id with fingerprint=5f2a6092668762c2d66dd0af58e952f82d9320441620741bd9c8e82e949f056d I0506 12:47:48.715771 1 gather.go:177] gatherer "clusterconfig" function "version" took 95.020863ms to process 2 records I0506 12:47:48.716849 1 tasks_processing.go:74] worker 61 stopped. I0506 12:47:48.717334 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=afb61dbcb59739c45bd6d529cfce5e6aeeaccb5e2b983f1ae012cb0d40e9773b I0506 12:47:48.717539 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=1a386010b8541dddd291a253d449ffdbe554a11956e95145009c50c1bcec655b I0506 12:47:48.717550 1 gather.go:177] gatherer "clusterconfig" function "crds" took 93.45811ms to process 2 records I0506 12:47:48.721964 1 tasks_processing.go:74] worker 31 stopped. I0506 12:47:48.721989 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 102.206129ms to process 0 records I0506 12:47:48.738152 1 tasks_processing.go:74] worker 1 stopped. I0506 12:47:48.739423 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt with fingerprint=4666cce1a8e9a1e8ef41b4d730c123703f7a4a6b365f6e43d8596ad46229e312 I0506 12:47:48.739497 1 recorder.go:75] Recording config/running_containers with fingerprint=93ecafb5a078b4f0d89b76de45f66af514878f9dc3f46b8fae5bb3e0a8808fe6 I0506 12:47:48.739507 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 118.302374ms to process 2 records I0506 12:47:48.739892 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0506 12:47:48.742908 1 tasks_processing.go:74] worker 5 stopped. I0506 12:47:48.742925 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 119.519472ms to process 0 records I0506 12:47:48.743417 1 base_controller.go:82] Caches are synced for LoggingSyncer I0506 12:47:48.743432 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... W0506 12:47:48.743626 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:54837->172.30.0.10:53: read: connection refused I0506 12:47:48.743638 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.10:54837->172.30.0.10:53: read: connection refused I0506 12:47:48.750070 1 tasks_processing.go:74] worker 34 stopped. E0506 12:47:48.750083 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0506 12:47:48.750090 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0506 12:47:48.750102 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0506 12:47:48.750117 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=ad286723f58bdcfc37aeba3ec5b4110c08e1af59cd34d14b4bfaab02d18e4856 I0506 12:47:48.750158 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0506 12:47:48.750201 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=2ab5ab7b1b10d7fcf1197bb24dea7c90f400e4effc18c7356873209d54fdf84b I0506 12:47:48.750209 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0506 12:47:48.750219 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0506 12:47:48.750258 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0506 12:47:48.750265 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0506 12:47:48.750270 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 122.79738ms to process 7 records I0506 12:47:48.793126 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0506 12:47:48.798322 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0506 12:47:48.810297 1 tasks_processing.go:74] worker 17 stopped. E0506 12:47:48.810312 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0506 12:47:48.810318 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2q4ebqgk4rgc7q2jo863v7joabv9hqlm-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2q4ebqgk4rgc7q2jo863v7joabv9hqlm-primary-cert-bundle-secret" not found I0506 12:47:48.810372 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=c42cc82b3b85022787be21ef3defa5ef0637e64219deacf8be09461307cfe27e I0506 12:47:48.810384 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 190.491298ms to process 1 records W0506 12:47:49.656939 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 12:47:50.528509 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found W0506 12:47:50.656977 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 12:47:50.664692 1 gather_cluster_operator_pods_and_events.go:121] Found 46 pods with 94 containers I0506 12:47:50.664705 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 267721 bytes I0506 12:47:50.664779 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:50.871677 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:51.075896 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:51.127772 1 tasks_processing.go:74] worker 47 stopped. I0506 12:47:51.127930 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=afbc494ffb85807bce830d4e5fdd3536d2dc58fef698fd7493c0481819146c66 I0506 12:47:51.127994 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0506 12:47:51.128050 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=23f7d68835082898486be30822aecbbf170a487b293f71edd7a9e111d35e6da0 I0506 12:47:51.128109 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0506 12:47:51.128216 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=621008985e1f4838c1c37fe68c6d7fd1de98daa38860e415e95e9c6976525cec I0506 12:47:51.128239 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0506 12:47:51.128269 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=7b50b388a120c3d0ab1d16efcc84c8b88f9442b42c5a7345ee2a2252575c8513 I0506 12:47:51.128310 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=726a77d7c591420406e6168caa743b65ce33ff641eb23b1ffac258c623b1d0fd I0506 12:47:51.128348 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=4e2b91daea34aa15ff968de4ec7d18ded6ae308ff191d69911d6edd1b1f480b4 I0506 12:47:51.128429 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=93ae24c83c564615d9ed6bac061546dbf2d8cbddb34ae71a132a0cb14f749a91 I0506 12:47:51.128725 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=5c388977183782b43801844c5d110f240910cf5ecdb5b78e8d572c0ac0de31a7 I0506 12:47:51.128757 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=fbb5199ce86ba5afde205b41a20bd40e058de98d54da1d1140378ec46ea04b81 I0506 12:47:51.128772 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0506 12:47:51.128792 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=02708d9b9e3c8ae12b7c380d30c4dbc196f1b54d71d832b0ed70d314fcd8f4e7 I0506 12:47:51.128803 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0506 12:47:51.128819 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=287e0ac84e047e7089536eeb35a191a6866206d29a76bb987077b5d4d75c2789 I0506 12:47:51.128828 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0506 12:47:51.128851 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=363f779aff572b4d6540f5a9933decdb27f2054e6250910669e41f461aad5716 I0506 12:47:51.128865 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0506 12:47:51.128897 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=8affdf869a4818526135952e42f4100411fb9be2325da5e00ac222dfcdcd676e I0506 12:47:51.129036 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=3d792f15d957b1ceb264cce91d24d3fabf2d1a38e8fbaf4ef6fd7ba10c128987 I0506 12:47:51.129087 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0506 12:47:51.129121 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0506 12:47:51.129193 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0506 12:47:51.129259 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=1ee127d2eb67a06d945819143ca25f2622c7b2b8b006c64e280b97ae5df17068 I0506 12:47:51.129302 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=fd454937b0b0e1031505b755154a5bbab9eece77b5df3f38a0bcc075fb0cb51e I0506 12:47:51.129317 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0506 12:47:51.129345 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=660b16001a820c845ea577488709dc1e2cd02fc9fea08848920101e4f60a4136 I0506 12:47:51.129359 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0506 12:47:51.129396 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=7782959a3c936e6ece216f8c272193a3d00ca0e0ca3516ae4f3782041b5c1e8c I0506 12:47:51.129450 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=86c3075dc076b0433d87cf60058831716bbb60bc7e2e141d0e9bc8ceb71eb160 I0506 12:47:51.129476 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=6145ccfad43506380dc30d620185bbb148018f907a6e00c8e4793bdccca6609d I0506 12:47:51.129500 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=8fe5acfa3790642f6d48451b1c495c911188d214bb9683413e9e36c3af82974c I0506 12:47:51.129536 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=5f624134d1bb39f32ac55f27bfa562706242895c8b8ae4d393a6572806d5d7da I0506 12:47:51.129554 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0506 12:47:51.129592 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=efa1464be95be6ee4ec6b47dcb017e28998c996df229ac154eee981107a4e009 I0506 12:47:51.129621 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0506 12:47:51.129635 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0506 12:47:51.129647 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.506950257s to process 38 records I0506 12:47:51.305312 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:51.482527 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). W0506 12:47:51.656055 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 12:47:51.672379 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:51.873289 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:52.074240 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:52.297695 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for alertmanager container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:52.475807 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). W0506 12:47:52.656211 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0506 12:47:52.675846 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:52.872547 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:53.079494 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metric container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:53.272392 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container alertmanager-main-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:53.473415 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for cluster-monitoring-operator container cluster-monitoring-operator-557b74df6-s6qn7 pod in namespace openshift-monitoring (previous: false). W0506 12:47:53.652624 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0506 12:47:53.652652 1 tasks_processing.go:74] worker 30 stopped. E0506 12:47:53.652665 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0506 12:47:53.652676 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0506 12:47:53.652686 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.032877853s to process 1 records I0506 12:47:53.677347 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-state-metrics container kube-state-metrics-6dfffbb599-j8cvx pod in namespace openshift-monitoring (previous: false). I0506 12:47:53.871642 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container kube-state-metrics-6dfffbb599-j8cvx pod in namespace openshift-monitoring (previous: false). I0506 12:47:54.072053 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container kube-state-metrics-6dfffbb599-j8cvx pod in namespace openshift-monitoring (previous: false). I0506 12:47:54.271853 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-568b56c7cc-bfss7 pod in namespace openshift-monitoring (previous: false). I0506 12:47:54.470751 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for metrics-server container metrics-server-568b56c7cc-c85th pod in namespace openshift-monitoring (previous: false). I0506 12:47:54.670581 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-5cbb48bf6b-pxpgm pod in namespace openshift-monitoring (previous: false). I0506 12:47:54.870034 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for monitoring-plugin container monitoring-plugin-5cbb48bf6b-x2f62 pod in namespace openshift-monitoring (previous: false). I0506 12:47:55.081012 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-8d295 pod in namespace openshift-monitoring (previous: false). I0506 12:47:55.289059 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 12:47:55.289077 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-8d295 pod in namespace openshift-monitoring (previous: false). I0506 12:47:55.470764 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-8d295 pod in namespace openshift-monitoring (previous: false). I0506 12:47:55.670144 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-jxwdq pod in namespace openshift-monitoring (previous: false). I0506 12:47:55.870153 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 12:47:55.870181 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-jxwdq pod in namespace openshift-monitoring (previous: false). I0506 12:47:56.070328 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-jxwdq pod in namespace openshift-monitoring (previous: false). I0506 12:47:56.269507 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-textfile container node-exporter-th86g pod in namespace openshift-monitoring (previous: false). I0506 12:47:56.470140 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 12:47:56.470156 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-exporter container node-exporter-th86g pod in namespace openshift-monitoring (previous: false). I0506 12:47:56.670883 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container node-exporter-th86g pod in namespace openshift-monitoring (previous: false). I0506 12:47:56.869999 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-main container openshift-state-metrics-b8587444b-86sdj pod in namespace openshift-monitoring (previous: false). I0506 12:47:57.070939 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-self container openshift-state-metrics-b8587444b-86sdj pod in namespace openshift-monitoring (previous: false). I0506 12:47:57.271202 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for openshift-state-metrics container openshift-state-metrics-b8587444b-86sdj pod in namespace openshift-monitoring (previous: false). I0506 12:47:57.474212 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:57.673707 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:57.872203 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:58.070534 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:58.271993 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:58.470607 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:58.671831 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-0 pod in namespace openshift-monitoring (previous: false). I0506 12:47:58.870386 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for init-config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:59.071712 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:59.273317 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for config-reloader container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:59.470988 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-sidecar container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:59.671524 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:47:59.870324 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:48:00.071913 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-thanos container prometheus-k8s-1 pod in namespace openshift-monitoring (previous: false). I0506 12:48:00.270879 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator container prometheus-operator-76b589d668-jqz9r pod in namespace openshift-monitoring (previous: false). I0506 12:48:00.473983 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container prometheus-operator-76b589d668-jqz9r pod in namespace openshift-monitoring (previous: false). I0506 12:48:00.671494 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-59b79769fb-4jv2d pod in namespace openshift-monitoring (previous: false). I0506 12:48:00.871469 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prometheus-operator-admission-webhook container prometheus-operator-admission-webhook-59b79769fb-tf9gr pod in namespace openshift-monitoring (previous: false). I0506 12:48:01.073407 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-6c5c77d46d-6g7pf pod in namespace openshift-monitoring (previous: false). I0506 12:48:01.277362 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-6c5c77d46d-6g7pf pod in namespace openshift-monitoring (previous: false). I0506 12:48:01.463945 1 tasks_processing.go:74] worker 63 stopped. I0506 12:48:01.463991 1 recorder.go:75] Recording config/installplans with fingerprint=95dfd5f33a9a46199239158a0ec0183b629818eca24ae41f4d706d7a3d604aec I0506 12:48:01.464004 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.842970939s to process 1 records I0506 12:48:01.471305 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-6c5c77d46d-6g7pf pod in namespace openshift-monitoring (previous: false). I0506 12:48:01.671306 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-6c5c77d46d-6g7pf pod in namespace openshift-monitoring (previous: false). I0506 12:48:01.871221 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-6c5c77d46d-6g7pf pod in namespace openshift-monitoring (previous: false). I0506 12:48:02.071142 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-6c5c77d46d-6g7pf pod in namespace openshift-monitoring (previous: false). I0506 12:48:02.229892 1 tasks_processing.go:74] worker 16 stopped. I0506 12:48:02.230164 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=796bd9f20e825fd32af435d21e60c10c3ec4910fa1211c5a6338f5b81c3f8071 I0506 12:48:02.230203 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.609966727s to process 1 records I0506 12:48:02.270748 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for thanos-query container thanos-querier-6c5c77d46d-rfwf8 pod in namespace openshift-monitoring (previous: false). I0506 12:48:02.472118 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-web container thanos-querier-6c5c77d46d-rfwf8 pod in namespace openshift-monitoring (previous: false). I0506 12:48:02.673334 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container thanos-querier-6c5c77d46d-rfwf8 pod in namespace openshift-monitoring (previous: false). I0506 12:48:02.873081 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for prom-label-proxy container thanos-querier-6c5c77d46d-rfwf8 pod in namespace openshift-monitoring (previous: false). I0506 12:48:03.072942 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-rules container thanos-querier-6c5c77d46d-rfwf8 pod in namespace openshift-monitoring (previous: false). I0506 12:48:03.273366 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy-metrics container thanos-querier-6c5c77d46d-rfwf8 pod in namespace openshift-monitoring (previous: false). I0506 12:48:03.470506 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-7f449dfbfd-dff9d pod in namespace openshift-network-console (previous: false). I0506 12:48:03.671682 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for networking-console-plugin container networking-console-plugin-7f449dfbfd-vfrvj pod in namespace openshift-network-console (previous: false). I0506 12:48:03.870937 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-66dbbb6468-rc8zt pod in namespace openshift-console-operator (previous: true). I0506 12:48:04.072080 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console-operator container console-operator-66dbbb6468-rc8zt pod in namespace openshift-console-operator (previous: false). I0506 12:48:04.275408 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-8684f95fc6-44pzc pod in namespace openshift-console (previous: false). I0506 12:48:04.464145 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for console-8684f95fc6-44pzc pod in namespace openshift-console for failing operator console (previous: false): "pods \"console-8684f95fc6-44pzc\" not found" I0506 12:48:04.464161 1 gather_cluster_operator_pods_and_events.go:280] Error: "pods \"console-8684f95fc6-44pzc\" not found" I0506 12:48:04.464183 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-8684f95fc6-kbjk2 pod in namespace openshift-console (previous: false). I0506 12:48:04.671786 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-c77d57cf5-2srnj pod in namespace openshift-console (previous: false). I0506 12:48:04.871134 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for console container console-c77d57cf5-rxd4v pod in namespace openshift-console (previous: false). I0506 12:48:05.069662 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-5c54f4944f-m44sj pod in namespace openshift-console (previous: false). I0506 12:48:05.272592 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for download-server container downloads-5c54f4944f-scqpd pod in namespace openshift-console (previous: false). I0506 12:48:05.474415 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-dh8wd pod in namespace openshift-dns (previous: false). I0506 12:48:05.674239 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-dh8wd pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-dh8wd\" is waiting to start: ContainerCreating" I0506 12:48:05.674260 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-dh8wd\" is waiting to start: ContainerCreating" I0506 12:48:05.674268 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-dh8wd pod in namespace openshift-dns (previous: false). I0506 12:48:05.871568 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-dh8wd pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-dh8wd\" is waiting to start: ContainerCreating" I0506 12:48:05.871586 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-dh8wd\" is waiting to start: ContainerCreating" I0506 12:48:05.871615 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-qv7pj pod in namespace openshift-dns (previous: false). I0506 12:48:06.070079 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-qv7pj pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-qv7pj\" is waiting to start: ContainerCreating" I0506 12:48:06.070098 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-qv7pj\" is waiting to start: ContainerCreating" I0506 12:48:06.070106 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-qv7pj pod in namespace openshift-dns (previous: false). I0506 12:48:06.270766 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-qv7pj pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-qv7pj\" is waiting to start: ContainerCreating" I0506 12:48:06.270786 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-qv7pj\" is waiting to start: ContainerCreating" I0506 12:48:06.270815 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-v94b7 pod in namespace openshift-dns (previous: false). I0506 12:48:06.470197 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-v94b7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-v94b7\" is waiting to start: ContainerCreating" I0506 12:48:06.470214 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-v94b7\" is waiting to start: ContainerCreating" I0506 12:48:06.470222 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-v94b7 pod in namespace openshift-dns (previous: false). I0506 12:48:06.671565 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-v94b7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-v94b7\" is waiting to start: ContainerCreating" I0506 12:48:06.671582 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-v94b7\" is waiting to start: ContainerCreating" I0506 12:48:06.671592 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-9vz4m pod in namespace openshift-dns (previous: false). I0506 12:48:06.870528 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 12:48:06.870548 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-db6pr pod in namespace openshift-dns (previous: false). I0506 12:48:07.070815 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 12:48:07.070838 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-zxwpx pod in namespace openshift-dns (previous: false). I0506 12:48:07.270459 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0506 12:48:07.270476 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-54df9c4f97-css4q pod in namespace openshift-image-registry (previous: false). I0506 12:48:07.471384 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-54df9c4f97-dcqvz pod in namespace openshift-image-registry (previous: false). I0506 12:48:07.671609 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-6b97597b97-t4zbp pod in namespace openshift-image-registry (previous: false). I0506 12:48:07.888575 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-6b97597b97-t4zbp pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-6b97597b97-t4zbp\" is waiting to start: ContainerCreating" I0506 12:48:07.888600 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-6b97597b97-t4zbp\" is waiting to start: ContainerCreating" I0506 12:48:07.888614 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-6w4c6 pod in namespace openshift-image-registry (previous: false). I0506 12:48:08.071622 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-bxfv2 pod in namespace openshift-image-registry (previous: false). I0506 12:48:08.271660 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-lgj7k pod in namespace openshift-image-registry (previous: false). I0506 12:48:08.470492 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-bc886f5b4-gcrsv pod in namespace openshift-ingress (previous: false). I0506 12:48:08.671433 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-bc886f5b4-qqz9d pod in namespace openshift-ingress (previous: false). I0506 12:48:08.871789 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-bff596dbc-fdjpl pod in namespace openshift-ingress (previous: false). I0506 12:48:09.064658 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-bff596dbc-fdjpl pod in namespace openshift-ingress for failing operator router (previous: false): "pods \"router-default-bff596dbc-fdjpl\" not found" I0506 12:48:09.064675 1 gather_cluster_operator_pods_and_events.go:280] Error: "pods \"router-default-bff596dbc-fdjpl\" not found" I0506 12:48:09.064684 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-8nkw9 pod in namespace openshift-ingress-canary (previous: false). I0506 12:48:09.271546 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-fcbp2 pod in namespace openshift-ingress-canary (previous: false). I0506 12:48:09.475166 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-l86g5 pod in namespace openshift-ingress-canary (previous: false). I0506 12:48:09.675852 1 tasks_processing.go:74] worker 32 stopped. I0506 12:48:09.676555 1 recorder.go:75] Recording events/openshift-monitoring with fingerprint=de6e610985ec50a75d875308a268aad5d74cfb16ccc460e698f1d430f8770f93 I0506 12:48:09.676602 1 recorder.go:75] Recording events/openshift-network-console with fingerprint=e46802febdb0ad652607adedcaffbdfa10a6ca6de87d608bc7f890124b4f05b2 I0506 12:48:09.676715 1 recorder.go:75] Recording events/openshift-console-operator with fingerprint=7d13aacf1c44e564d25964821d47f859812331c1a87f13a7d7b9aebb18832ff0 I0506 12:48:09.676783 1 recorder.go:75] Recording events/openshift-console with fingerprint=a96bfcbdae4f69f872d01bbfd6148363cfb704a74455824f2f1169138cdd2e87 I0506 12:48:09.676809 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=2541ca1929dfeb6e13dc7913fda7dda843d8fad2e24acc0324b17d2e5b278e43 I0506 12:48:09.676852 1 recorder.go:75] Recording events/openshift-dns with fingerprint=daf257ee1723483703b6b13bd3f6dbbaed67953678da359cf4fd4692bcb9a496 I0506 12:48:09.676974 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=356273e4de275549d38b37088e9e8ed32b3bb7e73d851228197e04320c980e42 I0506 12:48:09.677002 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=5e90d3f2347afa34535e75c8d1262dd8328100a1c45527fee1d19b34dc959c54 I0506 12:48:09.677069 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=bea4213d6ef77602bf4ebc867e9d9a21bedc5a914d7f0b2b93c105bc61d8f411 I0506 12:48:09.677135 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=cdd84d4ba4634d26e51d21c6256bf381f7397195fda4d93e938487152f325a8a I0506 12:48:09.677146 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/init-config-reloader_current.log with fingerprint=ef58a7dc4bf7145b0db6033a11d6fb651fabfd8793cd19fc46962b2ac0af4a3f I0506 12:48:09.677169 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/alertmanager_current.log with fingerprint=43155361f616c243a471ae8789772024d108d513c8445564cfd2f78e9e3cd4c8 I0506 12:48:09.677194 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/config-reloader_current.log with fingerprint=cbe1c70a0e873f18547f18b87ec365d8bfc833629240f1f3fe8c888df3583991 I0506 12:48:09.677204 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-web_current.log with fingerprint=c1d6d24c11d924e33f01d2328a00560aab91026e41c2bde6d1ad8e8e6c35c7a7 I0506 12:48:09.677212 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy_current.log with fingerprint=614f45e0fb992f42f0dc10b93f2837130dc7630d6b4700704f96ea5361663363 I0506 12:48:09.677218 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/kube-rbac-proxy-metric_current.log with fingerprint=afe713915f61c26b7aa9fb57197a5e1819fcababedcd4f2c7917b06f919f0322 I0506 12:48:09.677223 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-0/prom-label-proxy_current.log with fingerprint=743542d1021cb69640e8bca7558971d4373e6879c43f1178324cb40c1e966496 I0506 12:48:09.677230 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/init-config-reloader_current.log with fingerprint=d84b0f38597c6b498f54323cc830b061e8d41b20c3af700e42b1bc4208beb1f1 I0506 12:48:09.677252 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/alertmanager_current.log with fingerprint=42528e8a66f5f3f2212a0d9560bcd2566861bd857d0876ece862464f7bebe260 I0506 12:48:09.677267 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/config-reloader_current.log with fingerprint=bb9c6874ff95c6bb1033417620ebf90b208f3662aab3d710fba85846d927f884 I0506 12:48:09.677276 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-web_current.log with fingerprint=d7b99f8a942552861648ba2a7324bac4422fd729d6b0685f0233d70a86e2274d I0506 12:48:09.677283 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy_current.log with fingerprint=76495c8589ca2ca21eeb53443e7c91af2029b7135d897dfd279327651a56f000 I0506 12:48:09.677288 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/kube-rbac-proxy-metric_current.log with fingerprint=8b66a8a064f0a6f0239822b1c70e96ecdc5343cbe3afd3aaada43574d2d02304 I0506 12:48:09.677293 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/alertmanager-main-1/prom-label-proxy_current.log with fingerprint=2e502fb66d38bb0c866fdfd5be4f92ea491332787223e3fa70b9b988700a4f22 I0506 12:48:09.677411 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/cluster-monitoring-operator-557b74df6-s6qn7/cluster-monitoring-operator_current.log with fingerprint=09060d0f44235a7a61fa2d26a7ae29e97ac884136e7b3ee3a8fce7af38707acb I0506 12:48:09.677437 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-6dfffbb599-j8cvx/kube-state-metrics_current.log with fingerprint=9da02f90b9246f51d5f19eabd76cdab12b2796225d68ce76a33e46be124d9be6 I0506 12:48:09.677446 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-6dfffbb599-j8cvx/kube-rbac-proxy-main_current.log with fingerprint=4e1b589c411eeaa4b2666eb845ec3280ca9a6bd71bedf2e72f3d9514d81e5acd I0506 12:48:09.677452 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/kube-state-metrics-6dfffbb599-j8cvx/kube-rbac-proxy-self_current.log with fingerprint=7ca52cfb05411cea41db47fdfef04ef090b09a80e7b3f6074d35df3cc146e7e3 I0506 12:48:09.677461 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-568b56c7cc-bfss7/metrics-server_current.log with fingerprint=56cfb3fa2c24f1b804daa6d966fa413efea81b5fcf1ee4545879bcdcab84b12b I0506 12:48:09.677468 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/metrics-server-568b56c7cc-c85th/metrics-server_current.log with fingerprint=70d5dc4d7818a6fac32c23606c3c52899d766e54fca030a64c21a32515f3babf I0506 12:48:09.677478 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-5cbb48bf6b-pxpgm/monitoring-plugin_current.log with fingerprint=68a29cb0a7320fe187b1ce5c278915cc00329c9a4682983dac6cb7409dae8f6d I0506 12:48:09.677489 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/monitoring-plugin-5cbb48bf6b-x2f62/monitoring-plugin_current.log with fingerprint=6317ef4baee08599878b4f186d7dbe8eeb18b0cf2a9e70b5fd1c72bc74f93b21 I0506 12:48:09.677515 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-8d295/node-exporter_current.log with fingerprint=695164cacdd483feaf25edc2c1203fc13223875fdb68768625e15ae573c46379 I0506 12:48:09.677524 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-8d295/kube-rbac-proxy_current.log with fingerprint=f31f2b1aed15a7deda9c486470c3ef6333da44304b54c68f1c75312f96ccb4c1 I0506 12:48:09.677551 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-jxwdq/node-exporter_current.log with fingerprint=fdce435078357ad8ba1494f694379d345739d62d6f578620aa986680cb21d7e5 I0506 12:48:09.677559 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-jxwdq/kube-rbac-proxy_current.log with fingerprint=d55d150643998ea63c8bc3580dd83334c7ca87123fb0c1d1cbb3658535f0c9d1 I0506 12:48:09.677587 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-th86g/node-exporter_current.log with fingerprint=e479e5860f313e52961158c848b3522b5e82bd44646d527c5d94d16e26ffb4e1 I0506 12:48:09.677595 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/node-exporter-th86g/kube-rbac-proxy_current.log with fingerprint=eea673528e20d0b562c57a6270c5054a50bd4fd959fedbbd6913cf0a564e6cf7 I0506 12:48:09.677600 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-b8587444b-86sdj/kube-rbac-proxy-main_current.log with fingerprint=438b6733ec4ee2e4f6171499684943b6d0b5c027b320f95942dff4ed7cc6c5ea I0506 12:48:09.677606 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-b8587444b-86sdj/kube-rbac-proxy-self_current.log with fingerprint=e91e09bd53b551a2be7cad9f6f8ce2cd63f301cc925dc51ec541264e658b2bf2 I0506 12:48:09.677617 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/openshift-state-metrics-b8587444b-86sdj/openshift-state-metrics_current.log with fingerprint=a995af0f2578ef89b80b2ebfde9b464325abd7c861f09e391e152b1e075fdd4e I0506 12:48:09.677631 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/init-config-reloader_current.log with fingerprint=3d8a25efbf84ebf58a479b331990c6f2173697472254147019c227035c6939f0 I0506 12:48:09.677704 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/prometheus_current.log with fingerprint=fc7c1f3c23635c54404098098699ef4fe9defd199c0fa615110d8fdc961623bf I0506 12:48:09.677720 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/config-reloader_current.log with fingerprint=ef27e728751ce507642e7043d0efb4af1b73e65c44385c819e03a2f17cb9df75 I0506 12:48:09.677731 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/thanos-sidecar_current.log with fingerprint=aec16cb4304af3ddbbb26ff6f462e134b2a16a17d9126bf593ee64fd2fcc4258 I0506 12:48:09.677736 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-web_current.log with fingerprint=d915b3b56a480f3d38500a0d13ad41c87d27a1fcf3c1b12f480f00a9178102b9 I0506 12:48:09.677743 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy_current.log with fingerprint=8015e012c1fbff5ae3613791051d68c8a21a5627a8e02636cb8607fbf83465e7 I0506 12:48:09.677750 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-0/kube-rbac-proxy-thanos_current.log with fingerprint=dfc530e4832fa43034b142f67db784361cc9d4499e0cc6ac12387d93ab471de7 I0506 12:48:09.677756 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/init-config-reloader_current.log with fingerprint=473c689a89e732b9f35324a5a81917d6e389d0ae014bc6d121936bc3fc323cb1 I0506 12:48:09.677827 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/prometheus_current.log with fingerprint=1141e8387dd14640f4c5b7d99a3e27716b0383e96abeb48c5f494c51380c1ef4 I0506 12:48:09.677841 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/config-reloader_current.log with fingerprint=decc62ff315eeee354d3b72d2b4a1f5723a2fbc2fa39d6857925d7c0e70db4ea I0506 12:48:09.677853 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/thanos-sidecar_current.log with fingerprint=c481b318c3c9bdf8060e3465b5d2e46282636850f6faf1a003ea5a30f752afc3 I0506 12:48:09.677860 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-web_current.log with fingerprint=05dfcf263c06aed6a5382569a13249b8214fc69ece047b68bca8f5f316d52cd4 I0506 12:48:09.677866 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy_current.log with fingerprint=2b08c9756a8c56ee91c188292790019af822165b70c5aa695051ffc0cf8e47fb I0506 12:48:09.677872 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-k8s-1/kube-rbac-proxy-thanos_current.log with fingerprint=cf576244963dbb75267bdd34ef6ea152bd8ee99f98530fe9cf722871ce9a4ff6 I0506 12:48:09.677950 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-76b589d668-jqz9r/prometheus-operator_current.log with fingerprint=07a77df557b35947d021c9fa508d0990ce4e763860e208d53df0dcab1d76c0ae I0506 12:48:09.677958 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-76b589d668-jqz9r/kube-rbac-proxy_current.log with fingerprint=c2545bdf5c2d6f705bb57dd675bd976ee33f9f54a23f4e4c119ddaab0821184f I0506 12:48:09.677966 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-59b79769fb-4jv2d/prometheus-operator-admission-webhook_current.log with fingerprint=df026a35600efb26aac52b1df776b6a92387db529d6374991bd999dd54b5266b I0506 12:48:09.677973 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/prometheus-operator-admission-webhook-59b79769fb-tf9gr/prometheus-operator-admission-webhook_current.log with fingerprint=357f73080e196fd4ab5d7cb2270c99c846e924e695a2c6fab009e3be4795d875 I0506 12:48:09.678003 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-6g7pf/thanos-query_current.log with fingerprint=5e4dd4cbcae2e26ce2e26d421c00d359e3520531bb30c04146a861d9b18ac7a8 I0506 12:48:09.678020 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-6g7pf/kube-rbac-proxy-web_current.log with fingerprint=614ba63ae0d62e4378e93f622419bf032d85976debf8159762e4fff11c528bf7 I0506 12:48:09.678028 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-6g7pf/kube-rbac-proxy_current.log with fingerprint=6df1c40394c9099ce062907894241e305fdf8e720f7a95692f2a2806124b2b10 I0506 12:48:09.678032 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-6g7pf/prom-label-proxy_current.log with fingerprint=2d95190a695b96f6312e62dec94d6ce31a4b3a97ce6fb089fc09fa2aa5755d7f I0506 12:48:09.678038 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-6g7pf/kube-rbac-proxy-rules_current.log with fingerprint=7a1032650e817c1326d5b9125e7a7eba5423e4a1d41e724ee16cd0069b769eae I0506 12:48:09.678045 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-6g7pf/kube-rbac-proxy-metrics_current.log with fingerprint=5363bdb4f551e4396c86d7d88713232c600b38a3496cfda10cc3a37b79b13ef4 I0506 12:48:09.678074 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-rfwf8/thanos-query_current.log with fingerprint=9f4049f0cb105f21953c5b7e138ad8b9463984ea3b3aea54f4cdb4c75641872a I0506 12:48:09.678090 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-rfwf8/kube-rbac-proxy-web_current.log with fingerprint=444236b4129501c633f00f2678808ef0659115aab67187d12aa5b411783b73a8 I0506 12:48:09.678095 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-rfwf8/kube-rbac-proxy_current.log with fingerprint=93d41b3b1616a39ee7c9779d40589aa5eef963edac67dc436a2b1e739f0882b0 I0506 12:48:09.678099 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-rfwf8/prom-label-proxy_current.log with fingerprint=a56c303b4ce85d82224bae191680df6c02504f1b9f6115825216d6d652cda71e I0506 12:48:09.678105 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-rfwf8/kube-rbac-proxy-rules_current.log with fingerprint=2bb5346b626d164e58125632cc75be70128198490ffa15a90e30eda865c051cf I0506 12:48:09.678111 1 recorder.go:75] Recording config/pod/openshift-monitoring/logs/thanos-querier-6c5c77d46d-rfwf8/kube-rbac-proxy-metrics_current.log with fingerprint=d4c265b60bfed072db7794ba731d1326568033dcbdc76211a87c1faf9a592d68 I0506 12:48:09.678119 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-7f449dfbfd-dff9d/networking-console-plugin_current.log with fingerprint=8f0fa264f0f4e9698101ea8da96f11eecc7f007352254f29acf6f0fc2449b908 I0506 12:48:09.678124 1 recorder.go:75] Recording config/pod/openshift-network-console/logs/networking-console-plugin-7f449dfbfd-vfrvj/networking-console-plugin_current.log with fingerprint=a01b697f3d0892485d42d046b81f495281e22edc36cb7dd72e9edf606b83bd2e I0506 12:48:09.678269 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt with fingerprint=4666cce1a8e9a1e8ef41b4d730c123703f7a4a6b365f6e43d8596ad46229e312 E0506 12:48:09.678293 1 gather.go:161] error recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json" because of the error: the record with the same name "config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json" was already recorded and had the fingerprint "4666cce1a8e9a1e8ef41b4d730c123703f7a4a6b365f6e43d8596ad46229e312", overwriting with the record having fingerprint "4666cce1a8e9a1e8ef41b4d730c123703f7a4a6b365f6e43d8596ad46229e312" W0506 12:48:09.678305 1 gather.go:155] issue recording gatherer "clusterconfig" function "operators_pods_and_events" result "config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json" because of the warning: warning: the record with the same fingerprint "4666cce1a8e9a1e8ef41b4d730c123703f7a4a6b365f6e43d8596ad46229e312" was already recorded at path "config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json", recording another one with a different path "config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json" I0506 12:48:09.678327 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-66dbbb6468-rc8zt/console-operator_previous.log with fingerprint=59c8370bedd540c69b6379b19431a070d9da689c344c86d4b0b8f9431fba2a6e I0506 12:48:09.678553 1 recorder.go:75] Recording config/pod/openshift-console-operator/logs/console-operator-66dbbb6468-rc8zt/console-operator_current.log with fingerprint=f8e530c76fb4ba7d90b352badf5c77e893f3c084c923fcee46e0a7c5b3aa76c2 I0506 12:48:09.678565 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-8684f95fc6-kbjk2/console_current.log with fingerprint=749361ddf05d338577283aa86c313da43c3ce49cb1da187ebe101d4e1a4a6899 I0506 12:48:09.678575 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-c77d57cf5-2srnj/console_current.log with fingerprint=f3616e72a1fbda9201e1109a6993e8f2f1be4bfe0e0a0dea5ad90735510f7ad8 I0506 12:48:09.678586 1 recorder.go:75] Recording config/pod/openshift-console/logs/console-c77d57cf5-rxd4v/console_current.log with fingerprint=629909d2982f58a62858dd28f024142555d721f0f301d5ef96aac49a8d8a590d I0506 12:48:09.678614 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-5c54f4944f-m44sj/download-server_current.log with fingerprint=893e71a89006cec76e2ba89e351a525fb0c3fba42948a5731b192809ca410fb3 I0506 12:48:09.678644 1 recorder.go:75] Recording config/pod/openshift-console/logs/downloads-5c54f4944f-scqpd/download-server_current.log with fingerprint=0f0862d076e20c3a4e57e2b8d585bdc6352a3d1804e0299fa92990e07bdabb6e I0506 12:48:09.678744 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-dh8wd with fingerprint=82262aed4929e62eb7306d2252f1a7dddc27abb0f243613356d62e1b4d7c049d I0506 12:48:09.678814 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-qv7pj with fingerprint=ec9617acf7ed18b588aade62ff21b35633b72b03b71cc9a1877fa80f1064b933 I0506 12:48:09.678883 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-v94b7 with fingerprint=1c8d52b3fdc5e0ee7e6acfb8ea8c72cdb176f3b7471a3e7b061ad374a8ddf44c I0506 12:48:09.678899 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-54df9c4f97-css4q/registry_current.log with fingerprint=9ad80359727da5e5a287fb50a54413ea5aecca189e11429c50505b9ef60f5bad I0506 12:48:09.678915 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/image-registry-54df9c4f97-dcqvz/registry_current.log with fingerprint=325494b9242e105a100a8e888c6ba464ad7d904a99dda4fc4800c488e2cfb858 I0506 12:48:09.679012 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-6b97597b97-t4zbp with fingerprint=9fcd4927436eb6d1a724a85b0fa1d91c89eb501eeaa466af4752df9ec85a442a I0506 12:48:09.679020 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-6w4c6/node-ca_current.log with fingerprint=e6aa05471ae339a0be7932b18c252b08706760b8333cb5117f469595f06dbc32 I0506 12:48:09.679025 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-bxfv2/node-ca_current.log with fingerprint=0e44412e6039e1c32e66a7c05b867f042a062cd6808f430338cb7db0ca704c5a I0506 12:48:09.679029 1 recorder.go:75] Recording config/pod/openshift-image-registry/logs/node-ca-lgj7k/node-ca_current.log with fingerprint=85bb1935bb578326bc0b3d4fad1f518a777b57d30c92a2fb37b1fca0b739a69c I0506 12:48:09.679049 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-bc886f5b4-gcrsv/router_current.log with fingerprint=1d55706f03f26f9ba82c80cfaf6a6de87fbacb27bd40c1801749dcd1aa78aa07 I0506 12:48:09.679070 1 recorder.go:75] Recording config/pod/openshift-ingress/logs/router-default-bc886f5b4-qqz9d/router_current.log with fingerprint=65b35bab5f7cbefd22f7d36041e64dde86da906bef15ef9240bf1aad1149ccf0 I0506 12:48:09.679076 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-8nkw9/serve-healthcheck-canary_current.log with fingerprint=47ecd97bb012f0818630a4d127cbefe8be18fd28c6c5d3bc876425c84e4e4e95 I0506 12:48:09.679080 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-fcbp2/serve-healthcheck-canary_current.log with fingerprint=423036516a6bf387a3753de413a3ac8ecc8cc3f2f9e4c1b1e4bf39f0806f24df I0506 12:48:09.679084 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-l86g5/serve-healthcheck-canary_current.log with fingerprint=8559f143ec7ba49e4adddb661fbad29e7a4885194574c7db56adb60a72ce093f I0506 12:48:09.679089 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 21.056096225s to process 95 records E0506 12:48:09.679150 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 21.059s with: function \"silenced_alerts\" failed with an error, function \"active_alerts\" failed with an error, function \"metrics\" failed with an error, function \"tsdb_status\" failed with an error, function \"support_secret\" failed with an error, function \"machines\" failed with an error, function \"machine_configs\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error, unable to record function \"operators_pods_and_events\" record \"config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json\"" I0506 12:48:09.680279 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "silenced_alerts" failed with an error, function "active_alerts" failed with an error, function "metrics" failed with an error, function "tsdb_status" failed with an error, function "support_secret" failed with an error, function "machines" failed with an error, function "machine_configs" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error, unable to record function "operators_pods_and_events" record "config/pod/openshift-console-operator/console-operator-66dbbb6468-rc8zt.json" I0506 12:48:09.680297 1 periodic.go:216] Running workloads gatherer I0506 12:48:09.680314 1 tasks_processing.go:45] number of workers: 2 I0506 12:48:09.680326 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 12:48:09.680327 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 12:48:09.680336 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0506 12:48:09.680331 1 tasks_processing.go:71] worker 1 working on workload_info task. I0506 12:48:09.687256 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-148.ec2.internal... I0506 12:48:09.687267 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-30.ec2.internal... I0506 12:48:09.687283 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-131.ec2.internal... I0506 12:48:09.707410 1 tasks_processing.go:74] worker 0 stopped. I0506 12:48:09.707426 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 27.059265ms to process 0 records I0506 12:48:14.201565 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0506 12:48:14.229668 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 24s for image data I0506 12:48:14.237459 1 gather_workloads_info.go:387] No image sha256:730d1b6988025bef0daa3a9a5d8467ec4a26b0382cc52f91c3375b4590d3518a (9ms) I0506 12:48:14.246897 1 gather_workloads_info.go:387] No image sha256:a043239802b3eb8b323d285193d2527fad0ecec98ca91d188a3472a2fac8ae04 (9ms) I0506 12:48:14.255211 1 gather_workloads_info.go:387] No image sha256:934d8e8c50f3c609b8eea80d1051111fe3d066fe8c65c79572072ae55fcb0a86 (8ms) I0506 12:48:14.263228 1 gather_workloads_info.go:387] No image sha256:5aaea0419169e55832cc27acfe0fe3b9513a343d6bdf71d3da1575ed322245d0 (8ms) I0506 12:48:14.271367 1 gather_workloads_info.go:387] No image sha256:d6fbe0075cbb12bfd287c973704fadf97154c7f73e370733d976a40835e9155a (8ms) I0506 12:48:14.279314 1 gather_workloads_info.go:387] No image sha256:5b6212b8f539f08e78417d8a4b7485ca0b4e7927cacd7b752742a28841bc8ccd (8ms) I0506 12:48:14.287444 1 gather_workloads_info.go:387] No image sha256:1eb6a6ee3c66ac15be6bc21bb6015c595c11c15b9dfd21bd9f5ce7f7b990ebaa (8ms) I0506 12:48:14.295296 1 gather_workloads_info.go:387] No image sha256:25c148fd380b1a9db3f6039d2e0eabc489a954921452391390ba9192b2325678 (8ms) I0506 12:48:14.303195 1 gather_workloads_info.go:387] No image sha256:572ebf7c7fab1a518582f3ca46c866730423cce6e8a19f614dc15fd61bf0c65a (8ms) I0506 12:48:14.311237 1 gather_workloads_info.go:387] No image sha256:637d41f067a5239096fc22b135181cda5113da833f1370e7a73965e83792e93a (8ms) I0506 12:48:14.338160 1 gather_workloads_info.go:387] No image sha256:000105ef5150e7079b90a613fb9e6193e2a6ef9b1908d2dce44f2395d4fd070f (27ms) I0506 12:48:14.438122 1 gather_workloads_info.go:387] No image sha256:48f5559439bea030a6423652da448c25fff0c62ba293eb521843c1a8eb89ddb0 (100ms) I0506 12:48:14.539597 1 gather_workloads_info.go:387] No image sha256:30e597ec5d6bb96ff70a4f8688c748b659cd4fd5d73d222e8701821d236795c5 (101ms) I0506 12:48:14.637649 1 gather_workloads_info.go:387] No image sha256:89277d8d4560d71db88c2dcc67c992a24544ca21810920b609c1d49d53b4a287 (98ms) I0506 12:48:14.739656 1 gather_workloads_info.go:387] No image sha256:ae7d3453fd734ecc865e5f9bb16f29244ebffe6291b77e1d4e496f71eb053174 (102ms) I0506 12:48:14.838357 1 gather_workloads_info.go:387] No image sha256:3bba1358d4a0ae878ff491c0c2cbfffe60649e110b40342b878fe8fa332f8858 (99ms) I0506 12:48:14.937689 1 gather_workloads_info.go:387] No image sha256:90a8ffd9643ebb16a6a8c04bb38cf9ed58903e9d3bf836c68f399193db5edaf6 (99ms) I0506 12:48:15.039000 1 gather_workloads_info.go:387] No image sha256:17aefdc2f4752c23dc8f0947bb01fc34635822eeb18f2d8a5eddf8211b84d450 (101ms) I0506 12:48:15.141272 1 gather_workloads_info.go:387] No image sha256:23204877f166455ce8f9de9b91a467eeda576f172d5f96124a68aa986c4775d6 (102ms) I0506 12:48:15.238403 1 gather_workloads_info.go:387] No image sha256:ab60623bb32f7e75fca71ef65137731cae347a21c7a4091dfd583fa00732721c (97ms) I0506 12:48:15.338873 1 gather_workloads_info.go:387] No image sha256:bdeb1c455cffccdb6c90db238e9d371de06340b1d6105d47f66d87293f878ac1 (100ms) I0506 12:48:15.438130 1 gather_workloads_info.go:387] No image sha256:52db05084f2a2e7d7bf4040e4865df88ec4074efbbe59a78d861be2c5ab2aa40 (99ms) I0506 12:48:15.538247 1 gather_workloads_info.go:387] No image sha256:9ef6dc311b880a6442b7a3a19006ddd955c7da36850bf000f5c9b68fd0862dd4 (100ms) I0506 12:48:15.637751 1 gather_workloads_info.go:387] No image sha256:084aa9b0f8a6d478549dc384d4e66da13ee9b25cc98531da861cc19dee2a9e8f (99ms) I0506 12:48:15.739090 1 gather_workloads_info.go:387] No image sha256:55b1db6038c5beaed54c626e3343b7a8589cc0be8dc41d1a66b4deab766ff520 (101ms) I0506 12:48:15.838713 1 gather_workloads_info.go:387] No image sha256:673ebc8cc22c56c8f410e011b2fa950d28cf7b6420e17fdb6580d6cb10523384 (100ms) I0506 12:48:15.937645 1 gather_workloads_info.go:387] No image sha256:5d6a2b8d2508c7da71192661e7c0dcf1335f48b6336a20189abffbe79eeff962 (99ms) I0506 12:48:16.038353 1 gather_workloads_info.go:387] No image sha256:08c5a78c8a5af04c549e2273aaf4bb452a75bf038d68aa9d01bb2aff66c30e90 (101ms) I0506 12:48:16.137591 1 gather_workloads_info.go:387] No image sha256:5ac9c549d65fc1d8bc900773bebee43e9192bcec1bb5fa46afb4597230c16ac7 (99ms) I0506 12:48:16.237711 1 gather_workloads_info.go:387] No image sha256:0acfe677ed58031f8381cb8a4f2d8917ec4cfa95485e2a46529dffae54c75ace (100ms) I0506 12:48:16.338660 1 gather_workloads_info.go:387] No image sha256:5ff204630794311b0b37fa7b197b933ad85d76a481bf7cdb3bcbada08f0cdcbf (101ms) I0506 12:48:16.441261 1 gather_workloads_info.go:387] No image sha256:0b2e6536a0ca3afc5e64a218087a6287c74b77a359075a385f1949ca5bd51b22 (103ms) I0506 12:48:16.538321 1 gather_workloads_info.go:387] No image sha256:6c6d412ccf0c3d6a14645a79df6c4658c0f8725d7954cb83c3d2f0116499345a (97ms) I0506 12:48:16.637787 1 gather_workloads_info.go:387] No image sha256:9d6d63dce784df5863e6a0ba4c11317a26c9d756d4e62d47759c97190a71cce5 (99ms) I0506 12:48:16.738012 1 gather_workloads_info.go:387] No image sha256:5940cc34b322325618929b1e0eb3864dae73d54c8f5b474e218dfb28123a61c7 (100ms) I0506 12:48:16.838997 1 gather_workloads_info.go:387] No image sha256:3380fecebaeee139c1607fbcd9346cebc92806679568e2244a8905ef0f52aad2 (101ms) I0506 12:48:16.938797 1 gather_workloads_info.go:387] No image sha256:6814f5fced0ef219d06374011c68a11a3da788a764a00a69ade435466d9ee240 (100ms) I0506 12:48:17.038818 1 gather_workloads_info.go:387] No image sha256:d64bea34bf3e1bb0b3a701c3ff14e66665afc1b050f28124ad7e6888eaec3a81 (100ms) I0506 12:48:17.138290 1 gather_workloads_info.go:387] No image sha256:875c77e5d144f03fb91d8cee0259f6966683ca88d1bf818dbf4652c16b70312c (99ms) I0506 12:48:17.239149 1 gather_workloads_info.go:387] No image sha256:5c6d21c3f97366bc7ab57031cc027b67405a684bd804ce364ed5998b0685eaca (101ms) I0506 12:48:17.340460 1 gather_workloads_info.go:387] No image sha256:8f881f6f0498eaba860d711dc8e9dc066db8728f22abedb82e1dc9a8e1c30046 (101ms) I0506 12:48:17.438457 1 gather_workloads_info.go:387] No image sha256:15e1b5f763553e86d0230a751b0783e3b990438fd07cf96eed164bccb72ed564 (98ms) I0506 12:48:17.537206 1 gather_workloads_info.go:387] No image sha256:0a4dfb8d4c1b3849319d45b4c54dff26a7238a2c08fcaa121f93073e95ab12e8 (99ms) I0506 12:48:17.637508 1 gather_workloads_info.go:387] No image sha256:5808401268394502d335281ea1a294b07210461b986b58f91d7d1f29c0029c6d (100ms) I0506 12:48:17.738368 1 gather_workloads_info.go:387] No image sha256:15928a5b803fa00781b408dbc4c3bfa7c5287b18584ba31f817def8a4430c725 (101ms) I0506 12:48:17.837297 1 gather_workloads_info.go:387] No image sha256:48e883a3932aea9457f55cd4628d21397d429d4828f42f95d1c903d9d5395bde (99ms) I0506 12:48:17.938336 1 gather_workloads_info.go:387] No image sha256:53ff8efdd8f563421ebc1859ab021a3bbc1306f82d4ff5d9c21bc60ec5f57898 (101ms) I0506 12:48:18.038000 1 gather_workloads_info.go:387] No image sha256:15677f0b70e6aa2dfaf088e45fc1a425c22bf6fda326b8116f87e88b6694dfab (100ms) I0506 12:48:18.038033 1 tasks_processing.go:74] worker 1 stopped. I0506 12:48:18.038347 1 recorder.go:75] Recording config/workload_info with fingerprint=666711894ca3397935ba7a651daf7aa6e67396a3fba6e7b1791ccf459ad63134 I0506 12:48:18.038364 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.357683131s to process 1 records I0506 12:48:18.038378 1 periodic.go:263] Periodic gather workloads completed in 8.358s I0506 12:48:18.038387 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0506 12:48:18.038392 1 periodic.go:216] Running conditional gatherer I0506 12:48:18.044812 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules I0506 12:48:18.051537 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.10:49132->172.30.0.10:53: read: connection refused I0506 12:48:18.051859 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer E0506 12:48:18.055097 1 conditional_gatherer.go:324] unable to update alerts cache: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?match%5B%5D=ALERTS%7Balertstate%3D%22firing%22%7D&query=ALERTS": dial tcp: lookup prometheus-k8s.openshift-monitoring.svc on 172.30.0.10:53: read udp 10.130.0.10:42341->172.30.0.10:53: read: connection refused I0506 12:48:18.055162 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0506 12:48:18.060971 1 conditional_gatherer.go:392] cluster version is '4.21.9' E0506 12:48:18.060985 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.060991 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.060995 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.061000 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.061003 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.061006 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.061009 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.061012 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0506 12:48:18.061015 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0506 12:48:18.061030 1 tasks_processing.go:45] number of workers: 3 I0506 12:48:18.061045 1 tasks_processing.go:69] worker 2 listening for tasks. I0506 12:48:18.061048 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0506 12:48:18.061060 1 tasks_processing.go:69] worker 0 listening for tasks. I0506 12:48:18.061063 1 tasks_processing.go:69] worker 1 listening for tasks. I0506 12:48:18.061079 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0506 12:48:18.061090 1 tasks_processing.go:74] worker 1 stopped. I0506 12:48:18.061076 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0506 12:48:18.061126 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0506 12:48:18.061138 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 938ns to process 1 records I0506 12:48:18.061163 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0506 12:48:18.061190 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.715µs to process 1 records I0506 12:48:18.061202 1 tasks_processing.go:74] worker 0 stopped. I0506 12:48:18.061337 1 tasks_processing.go:74] worker 2 stopped. I0506 12:48:18.061349 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 242.018µs to process 0 records I0506 12:48:18.061368 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.10:49132->172.30.0.10:53: read: connection refused I0506 12:48:18.061385 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0506 12:48:18.087409 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=4f27285e08f8ca05c814a1ae06660f36bc1346468a1d52eeea9a2fa2cf3086fb I0506 12:48:18.087537 1 diskrecorder.go:70] Writing 202 records to /var/lib/insights-operator/insights-2026-05-06-124818.tar.gz I0506 12:48:18.101378 1 diskrecorder.go:51] Wrote 202 records to disk in 13ms I0506 12:48:18.101405 1 periodic.go:285] Gathering cluster info every 2h0m0s I0506 12:48:18.101427 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0506 12:49:18.619737 1 diskrecorder.go:223] Found files to send: insights-2026-05-06-124818.tar.gz I0506 12:49:18.619770 1 insightsuploader.go:150] Checking archives to upload periodically every 15m7.359398566s I0506 12:49:18.619778 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0506 12:49:18.631668 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0506 12:49:18.986950 1 requests.go:87] Successfully reported id=2026-05-06T12:49:18Z x-rh-insights-request-id=1723c6521be847b8a98535882aa304db, wrote=105171 I0506 12:49:18.987020 1 insightsuploader.go:187] Uploaded report successfully in 367.231424ms I0506 12:49:18.987046 1 controller.go:129] Initializing last reported time to 2026-05-06T12:49:18Z I0506 12:49:18.987113 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0506 12:49:18.987121 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0506 12:49:18.987128 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0506 12:49:18.992830 1 controller.go:554] The operator is healthy I0506 12:49:48.650522 1 controller.go:554] The operator is healthy I0506 12:50:24.174756 1 insightsreport.go:137] Pulling report from smart-proxy I0506 12:50:24.174779 1 insightsreport.go:149] Retrieving report I0506 12:50:24.181770 1 requests.go:110] Retrieving report for cluster: 6309b07b-3e1f-48b4-ad06-4144fecb5ba4 I0506 12:50:24.181786 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/6309b07b-3e1f-48b4-ad06-4144fecb5ba4/reports I0506 12:50:24.184905 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/6309b07b-3e1f-48b4-ad06-4144fecb5ba4/reports I0506 12:50:24.490108 1 insightsreport.go:184] Report retrieved I0506 12:50:24.518307 1 insightsreport.go:239] Report retrieved correctly I0506 12:51:48.649951 1 controller.go:554] The operator is healthy I0506 12:52:48.612656 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 12:52:48.618599 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 12:52:48.618616 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 12:52:48.624545 1 secretconfigobserver.go:119] support secret does not exist I0506 12:53:48.650157 1 controller.go:554] The operator is healthy I0506 12:55:48.650360 1 controller.go:554] The operator is healthy I0506 12:57:48.609752 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 12:57:48.618030 1 configmapobserver.go:84] configmaps "insights-config" not found I0506 12:57:48.625123 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 12:57:48.639118 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 12:57:48.639133 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 12:57:48.670194 1 secretconfigobserver.go:119] support secret does not exist I0506 12:57:48.670519 1 controller.go:554] The operator is healthy I0506 12:59:48.650408 1 controller.go:554] The operator is healthy I0506 13:01:48.654800 1 controller.go:554] The operator is healthy I0506 13:02:48.670807 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0506 13:02:48.676152 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0506 13:02:48.676182 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0506 13:02:48.683780 1 secretconfigobserver.go:119] support secret does not exist I0506 13:03:48.653052 1 controller.go:554] The operator is healthy I0506 13:04:31.347556 1 insightsuploader.go:146] Nothing to report since 2026-05-06T12:49:18Z