W0419 18:57:37.834628 1 cmd.go:257] Using insecure, self-signed certificates I0419 18:57:38.236610 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0419 18:57:38.236947 1 observer_polling.go:159] Starting file observer I0419 18:57:38.513966 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0419 18:57:38.514153 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0419 18:57:38.514749 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0419 18:57:38.515215 1 secure_serving.go:57] Forcing use of http/1.1 only W0419 18:57:38.515241 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0419 18:57:38.515250 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0419 18:57:38.515256 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0419 18:57:38.515260 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0419 18:57:38.515264 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0419 18:57:38.515267 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0419 18:57:38.518795 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0419 18:57:38.519059 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"21d194fe-5193-4a3e-85eb-e1ba55865c03", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0419 18:57:38.520627 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0419 18:57:38.520650 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0419 18:57:38.520676 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0419 18:57:38.520676 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0419 18:57:38.520695 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0419 18:57:38.520705 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0419 18:57:38.520905 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-4086935872/tls.crt::/tmp/serving-cert-4086935872/tls.key" I0419 18:57:38.521203 1 secure_serving.go:213] Serving securely on [::]:8443 I0419 18:57:38.521238 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0419 18:57:38.524183 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0419 18:57:38.524206 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0419 18:57:38.524298 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0419 18:57:38.528665 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0419 18:57:38.528685 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0419 18:57:38.532999 1 secretconfigobserver.go:119] support secret does not exist I0419 18:57:38.537195 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0419 18:57:38.545029 1 secretconfigobserver.go:119] support secret does not exist I0419 18:57:38.548820 1 recorder.go:176] Pruning old reports every 8h38m31s, max age is 288h0m0s I0419 18:57:38.571849 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0419 18:57:38.571862 1 periodic.go:216] Running clusterconfig gatherer I0419 18:57:38.571872 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0419 18:57:38.571927 1 tasks_processing.go:45] number of workers: 64 I0419 18:57:38.571928 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0419 18:57:38.571941 1 insightsreport.go:296] Starting report retriever I0419 18:57:38.571948 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0419 18:57:38.571962 1 tasks_processing.go:69] worker 1 listening for tasks. I0419 18:57:38.571966 1 tasks_processing.go:69] worker 11 listening for tasks. I0419 18:57:38.571974 1 tasks_processing.go:69] worker 2 listening for tasks. I0419 18:57:38.571983 1 tasks_processing.go:69] worker 3 listening for tasks. I0419 18:57:38.571985 1 tasks_processing.go:69] worker 18 listening for tasks. I0419 18:57:38.571988 1 tasks_processing.go:69] worker 0 listening for tasks. I0419 18:57:38.571993 1 tasks_processing.go:71] worker 3 working on machine_config_pools task. I0419 18:57:38.571998 1 tasks_processing.go:69] worker 4 listening for tasks. I0419 18:57:38.572001 1 tasks_processing.go:71] worker 0 working on openshift_logging task. I0419 18:57:38.572006 1 tasks_processing.go:71] worker 4 working on cost_management_metrics_configs task. I0419 18:57:38.572009 1 tasks_processing.go:69] worker 20 listening for tasks. I0419 18:57:38.572012 1 tasks_processing.go:69] worker 31 listening for tasks. I0419 18:57:38.572015 1 tasks_processing.go:69] worker 21 listening for tasks. I0419 18:57:38.572004 1 tasks_processing.go:69] worker 42 listening for tasks. I0419 18:57:38.572021 1 tasks_processing.go:69] worker 22 listening for tasks. I0419 18:57:38.572021 1 tasks_processing.go:69] worker 8 listening for tasks. I0419 18:57:38.572026 1 tasks_processing.go:69] worker 23 listening for tasks. I0419 18:57:38.572029 1 tasks_processing.go:69] worker 5 listening for tasks. I0419 18:57:38.572034 1 tasks_processing.go:69] worker 24 listening for tasks. I0419 18:57:38.572035 1 tasks_processing.go:69] worker 54 listening for tasks. I0419 18:57:38.572040 1 tasks_processing.go:69] worker 7 listening for tasks. I0419 18:57:38.572035 1 tasks_processing.go:69] worker 6 listening for tasks. I0419 18:57:38.572039 1 tasks_processing.go:69] worker 32 listening for tasks. I0419 18:57:38.572043 1 tasks_processing.go:69] worker 25 listening for tasks. I0419 18:57:38.572048 1 tasks_processing.go:69] worker 33 listening for tasks. I0419 18:57:38.572050 1 tasks_processing.go:69] worker 43 listening for tasks. I0419 18:57:38.572051 1 tasks_processing.go:69] worker 26 listening for tasks. I0419 18:57:38.572057 1 tasks_processing.go:69] worker 44 listening for tasks. I0419 18:57:38.572057 1 tasks_processing.go:69] worker 60 listening for tasks. I0419 18:57:38.572065 1 tasks_processing.go:69] worker 12 listening for tasks. I0419 18:57:38.572066 1 tasks_processing.go:69] worker 28 listening for tasks. I0419 18:57:38.572056 1 tasks_processing.go:69] worker 34 listening for tasks. I0419 18:57:38.572073 1 tasks_processing.go:69] worker 63 listening for tasks. I0419 18:57:38.572064 1 tasks_processing.go:69] worker 35 listening for tasks. I0419 18:57:38.572058 1 tasks_processing.go:69] worker 27 listening for tasks. I0419 18:57:38.572077 1 tasks_processing.go:69] worker 14 listening for tasks. I0419 18:57:38.572080 1 tasks_processing.go:69] worker 59 listening for tasks. I0419 18:57:38.572087 1 tasks_processing.go:69] worker 29 listening for tasks. I0419 18:57:38.572071 1 tasks_processing.go:69] worker 36 listening for tasks. I0419 18:57:38.572092 1 tasks_processing.go:69] worker 13 listening for tasks. I0419 18:57:38.572073 1 tasks_processing.go:69] worker 56 listening for tasks. I0419 18:57:38.572100 1 tasks_processing.go:69] worker 41 listening for tasks. I0419 18:57:38.572073 1 tasks_processing.go:69] worker 45 listening for tasks. I0419 18:57:38.572066 1 tasks_processing.go:69] worker 55 listening for tasks. I0419 18:57:38.572107 1 tasks_processing.go:69] worker 52 listening for tasks. I0419 18:57:38.572107 1 tasks_processing.go:69] worker 57 listening for tasks. I0419 18:57:38.572110 1 tasks_processing.go:69] worker 40 listening for tasks. I0419 18:57:38.572119 1 tasks_processing.go:69] worker 9 listening for tasks. I0419 18:57:38.572119 1 tasks_processing.go:69] worker 10 listening for tasks. I0419 18:57:38.572120 1 tasks_processing.go:69] worker 17 listening for tasks. I0419 18:57:38.571999 1 tasks_processing.go:69] worker 30 listening for tasks. I0419 18:57:38.572080 1 tasks_processing.go:69] worker 37 listening for tasks. I0419 18:57:38.572080 1 tasks_processing.go:69] worker 46 listening for tasks. I0419 18:57:38.572084 1 tasks_processing.go:69] worker 61 listening for tasks. I0419 18:57:38.572086 1 tasks_processing.go:69] worker 38 listening for tasks. I0419 18:57:38.572088 1 tasks_processing.go:69] worker 47 listening for tasks. I0419 18:57:38.572094 1 tasks_processing.go:71] worker 11 working on image task. I0419 18:57:38.572003 1 tasks_processing.go:69] worker 19 listening for tasks. I0419 18:57:38.572373 1 tasks_processing.go:71] worker 47 working on proxies task. I0419 18:57:38.572391 1 tasks_processing.go:71] worker 19 working on certificate_signing_requests task. I0419 18:57:38.572430 1 tasks_processing.go:71] worker 54 working on storage_classes task. I0419 18:57:38.572433 1 tasks_processing.go:71] worker 6 working on sap_datahubs task. I0419 18:57:38.572446 1 tasks_processing.go:71] worker 35 working on openstack_dataplanedeployments task. I0419 18:57:38.572453 1 tasks_processing.go:71] worker 8 working on aggregated_monitoring_cr_names task. I0419 18:57:38.572468 1 tasks_processing.go:71] worker 44 working on machines task. I0419 18:57:38.572515 1 tasks_processing.go:71] worker 24 working on machine_configs task. I0419 18:57:38.572090 1 tasks_processing.go:69] worker 62 listening for tasks. I0419 18:57:38.572099 1 tasks_processing.go:69] worker 48 listening for tasks. I0419 18:57:38.572098 1 tasks_processing.go:71] worker 2 working on cluster_apiserver task. I0419 18:57:38.572621 1 tasks_processing.go:71] worker 7 working on tsdb_status task. I0419 18:57:38.572095 1 tasks_processing.go:69] worker 58 listening for tasks. I0419 18:57:38.573592 1 tasks_processing.go:71] worker 58 working on validating_webhook_configurations task. W0419 18:57:38.573601 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0419 18:57:38.572098 1 tasks_processing.go:71] worker 1 working on clusterroles task. I0419 18:57:38.573614 1 tasks_processing.go:71] worker 61 working on silenced_alerts task. I0419 18:57:38.573622 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 30.942µs to process 0 records I0419 18:57:38.572665 1 tasks_processing.go:71] worker 33 working on olm_operators task. W0419 18:57:38.573647 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0419 18:57:38.573663 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 31.536µs to process 0 records I0419 18:57:38.572102 1 tasks_processing.go:69] worker 51 listening for tasks. I0419 18:57:38.572109 1 tasks_processing.go:69] worker 50 listening for tasks. I0419 18:57:38.572110 1 tasks_processing.go:69] worker 16 listening for tasks. I0419 18:57:38.572110 1 tasks_processing.go:69] worker 49 listening for tasks. I0419 18:57:38.571995 1 tasks_processing.go:71] worker 18 working on pdbs task. I0419 18:57:38.572108 1 tasks_processing.go:69] worker 53 listening for tasks. I0419 18:57:38.572741 1 tasks_processing.go:71] worker 62 working on operators task. I0419 18:57:38.572679 1 tasks_processing.go:71] worker 48 working on nodenetworkstates task. I0419 18:57:38.572791 1 tasks_processing.go:71] worker 23 working on infrastructures task. I0419 18:57:38.573339 1 tasks_processing.go:71] worker 10 working on sap_config task. I0419 18:57:38.572759 1 tasks_processing.go:71] worker 32 working on machine_autoscalers task. I0419 18:57:38.574039 1 tasks_processing.go:71] worker 49 working on feature_gates task. I0419 18:57:38.572871 1 tasks_processing.go:71] worker 26 working on dvo_metrics task. I0419 18:57:38.573898 1 tasks_processing.go:71] worker 7 working on machine_sets task. I0419 18:57:38.572874 1 tasks_processing.go:71] worker 43 working on openstack_controlplanes task. I0419 18:57:38.572093 1 tasks_processing.go:69] worker 39 listening for tasks. I0419 18:57:38.575127 1 tasks_processing.go:71] worker 39 working on authentication task. I0419 18:57:38.572939 1 tasks_processing.go:71] worker 5 working on machine_healthchecks task. I0419 18:57:38.572958 1 tasks_processing.go:71] worker 21 working on container_images task. I0419 18:57:38.572975 1 tasks_processing.go:71] worker 20 working on openshift_machine_api_events task. I0419 18:57:38.572991 1 tasks_processing.go:71] worker 31 working on schedulers task. I0419 18:57:38.573004 1 tasks_processing.go:71] worker 57 working on container_runtime_configs task. I0419 18:57:38.573016 1 tasks_processing.go:71] worker 45 working on nodes task. I0419 18:57:38.573032 1 tasks_processing.go:71] worker 13 working on jaegers task. I0419 18:57:38.573067 1 tasks_processing.go:71] worker 55 working on active_alerts task. I0419 18:57:38.573100 1 tasks_processing.go:71] worker 27 working on metrics task. W0419 18:57:38.575433 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0419 18:57:38.575444 1 tasks_processing.go:71] worker 55 working on install_plans task. I0419 18:57:38.575448 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 25.693µs to process 0 records W0419 18:57:38.575452 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0419 18:57:38.575464 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.117µs to process 0 records I0419 18:57:38.573106 1 tasks_processing.go:71] worker 28 working on image_registries task. I0419 18:57:38.573122 1 tasks_processing.go:71] worker 52 working on node_logs task. I0419 18:57:38.573130 1 tasks_processing.go:71] worker 56 working on support_secret task. I0419 18:57:38.573168 1 tasks_processing.go:71] worker 60 working on networks task. I0419 18:57:38.575581 1 tasks_processing.go:74] worker 27 stopped. I0419 18:57:38.573181 1 tasks_processing.go:71] worker 14 working on mutating_webhook_configurations task. I0419 18:57:38.573191 1 tasks_processing.go:71] worker 12 working on pod_network_connectivity_checks task. I0419 18:57:38.573229 1 tasks_processing.go:71] worker 29 working on monitoring_persistent_volumes task. I0419 18:57:38.573255 1 tasks_processing.go:71] worker 36 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0419 18:57:38.573274 1 tasks_processing.go:71] worker 42 working on version task. I0419 18:57:38.573276 1 tasks_processing.go:71] worker 41 working on crds task. I0419 18:57:38.573288 1 tasks_processing.go:71] worker 34 working on nodenetworkconfigurationpolicies task. I0419 18:57:38.573313 1 tasks_processing.go:71] worker 22 working on qemu_kubevirt_launcher_logs task. I0419 18:57:38.573331 1 tasks_processing.go:71] worker 59 working on ingress_certificates task. I0419 18:57:38.573347 1 tasks_processing.go:71] worker 63 working on operators_pods_and_events task. I0419 18:57:38.573366 1 tasks_processing.go:71] worker 37 working on config_maps task. I0419 18:57:38.573395 1 tasks_processing.go:71] worker 17 working on service_accounts task. I0419 18:57:38.573409 1 tasks_processing.go:71] worker 30 working on overlapping_namespace_uids task. I0419 18:57:38.573414 1 tasks_processing.go:71] worker 40 working on storage_cluster task. I0419 18:57:38.573423 1 tasks_processing.go:71] worker 38 working on node_features task. I0419 18:57:38.573494 1 tasks_processing.go:69] worker 15 listening for tasks. I0419 18:57:38.576519 1 tasks_processing.go:74] worker 15 stopped. I0419 18:57:38.573499 1 tasks_processing.go:71] worker 46 working on openstack_dataplanenodesets task. I0419 18:57:38.573541 1 tasks_processing.go:71] worker 9 working on sap_pods task. I0419 18:57:38.572839 1 tasks_processing.go:71] worker 25 working on ceph_cluster task. I0419 18:57:38.573907 1 tasks_processing.go:71] worker 61 working on lokistack task. I0419 18:57:38.573875 1 tasks_processing.go:71] worker 53 working on oauths task. I0419 18:57:38.573856 1 tasks_processing.go:71] worker 51 working on openstack_version task. I0419 18:57:38.573923 1 tasks_processing.go:71] worker 50 working on image_pruners task. I0419 18:57:38.573930 1 tasks_processing.go:71] worker 16 working on ingress task. I0419 18:57:38.583178 1 tasks_processing.go:74] worker 4 stopped. I0419 18:57:38.583189 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 11.164731ms to process 0 records I0419 18:57:38.583196 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 11.172921ms to process 0 records I0419 18:57:38.583201 1 tasks_processing.go:74] worker 0 stopped. I0419 18:57:38.583207 1 tasks_processing.go:74] worker 3 stopped. I0419 18:57:38.583220 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 11.202266ms to process 0 records I0419 18:57:38.584876 1 tasks_processing.go:74] worker 35 stopped. I0419 18:57:38.584897 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 12.413373ms to process 0 records I0419 18:57:38.584909 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 12.424945ms to process 0 records I0419 18:57:38.584915 1 tasks_processing.go:74] worker 6 stopped. I0419 18:57:38.584997 1 tasks_processing.go:74] worker 44 stopped. E0419 18:57:38.585008 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0419 18:57:38.585017 1 gather.go:177] gatherer "clusterconfig" function "machines" took 12.508775ms to process 0 records I0419 18:57:38.591455 1 tasks_processing.go:74] worker 48 stopped. I0419 18:57:38.591464 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 17.735666ms to process 0 records I0419 18:57:38.591471 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 17.485557ms to process 0 records I0419 18:57:38.591475 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 17.424824ms to process 0 records I0419 18:57:38.591480 1 tasks_processing.go:74] worker 32 stopped. I0419 18:57:38.591483 1 tasks_processing.go:74] worker 10 stopped. I0419 18:57:38.591493 1 tasks_processing.go:74] worker 7 stopped. I0419 18:57:38.591498 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 16.72735ms to process 0 records I0419 18:57:38.591522 1 tasks_processing.go:74] worker 5 stopped. E0419 18:57:38.591535 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0419 18:57:38.591545 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 16.376344ms to process 0 records I0419 18:57:38.594536 1 tasks_processing.go:74] worker 11 stopped. I0419 18:57:38.594536 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0419 18:57:38.594555 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0419 18:57:38.594560 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0419 18:57:38.594565 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0419 18:57:38.594578 1 controller.go:531] The operator is still being initialized I0419 18:57:38.594584 1 controller.go:554] The operator is healthy I0419 18:57:38.594644 1 recorder.go:75] Recording config/image with fingerprint=d276454c842dfe834c94c1e96924d35d8541441bb979c39eb3177fedf88eef54 I0419 18:57:38.594660 1 gather.go:177] gatherer "clusterconfig" function "image" took 22.384399ms to process 1 records I0419 18:57:38.604873 1 tasks_processing.go:74] worker 2 stopped. I0419 18:57:38.605047 1 recorder.go:75] Recording config/apiserver with fingerprint=fdd45fb1be5aab7b4f86f6544781574bf4495b5c003ab66e53c4962764243f61 I0419 18:57:38.605064 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 32.252977ms to process 1 records I0419 18:57:38.605154 1 tasks_processing.go:74] worker 23 stopped. I0419 18:57:38.605517 1 recorder.go:75] Recording config/infrastructure with fingerprint=1768bb3f3727c298092241000e55d3ddef5bd71165c37ee7a9415d721dd4836f I0419 18:57:38.605532 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 31.11683ms to process 1 records I0419 18:57:38.605545 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 31.290606ms to process 0 records I0419 18:57:38.605597 1 recorder.go:75] Recording config/proxy with fingerprint=6b949b563f5843dd313ec28cbcf53aade6ffd2c6690e3d679a4081bb936495af I0419 18:57:38.605609 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 32.635003ms to process 1 records I0419 18:57:38.605615 1 tasks_processing.go:74] worker 33 stopped. I0419 18:57:38.605619 1 tasks_processing.go:74] worker 47 stopped. I0419 18:57:38.606492 1 tasks_processing.go:74] worker 40 stopped. I0419 18:57:38.606504 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 30.135799ms to process 0 records I0419 18:57:38.616822 1 tasks_processing.go:74] worker 56 stopped. E0419 18:57:38.616834 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0419 18:57:38.616840 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 41.271422ms to process 0 records I0419 18:57:38.616876 1 tasks_processing.go:74] worker 54 stopped. I0419 18:57:38.616982 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=383ac3c3fa8daa8b2611f2405df09bca701c40e8cc2c95cf09cdf1f6575ffb96 I0419 18:57:38.617002 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=fa1420d978dc308bcf10d6e92963263db554bd17a93c8831d783452aa0f4db4e I0419 18:57:38.617008 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 44.428231ms to process 2 records I0419 18:57:38.617061 1 tasks_processing.go:74] worker 43 stopped. I0419 18:57:38.617070 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 41.936644ms to process 0 records I0419 18:57:38.617076 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 40.53806ms to process 0 records I0419 18:57:38.617081 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 41.769371ms to process 0 records I0419 18:57:38.617085 1 tasks_processing.go:74] worker 57 stopped. I0419 18:57:38.617089 1 tasks_processing.go:74] worker 46 stopped. I0419 18:57:38.617095 1 tasks_processing.go:74] worker 34 stopped. I0419 18:57:38.617106 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 41.137935ms to process 0 records E0419 18:57:38.617116 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0419 18:57:38.617123 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 41.381735ms to process 0 records I0419 18:57:38.617130 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 40.556621ms to process 0 records I0419 18:57:38.617155 1 tasks_processing.go:74] worker 12 stopped. I0419 18:57:38.617155 1 tasks_processing.go:74] worker 9 stopped. I0419 18:57:38.617229 1 tasks_processing.go:74] worker 60 stopped. I0419 18:57:38.617281 1 recorder.go:75] Recording config/network with fingerprint=d9137bdda2307825b124cb37093d1130d5f7501a6cc2778244b013b7493ae98d I0419 18:57:38.617296 1 gather.go:177] gatherer "clusterconfig" function "networks" took 41.53616ms to process 1 records I0419 18:57:38.617304 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 40.534212ms to process 0 records I0419 18:57:38.617309 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 41.950538ms to process 0 records I0419 18:57:38.617314 1 tasks_processing.go:74] worker 61 stopped. E0419 18:57:38.617318 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0419 18:57:38.617322 1 tasks_processing.go:74] worker 13 stopped. I0419 18:57:38.617389 1 tasks_processing.go:74] worker 39 stopped. I0419 18:57:38.617472 1 recorder.go:75] Recording config/authentication with fingerprint=f03bfc620e3d8d57f0aca45a8ba5dc4a6c8d65beff68e63d6ad74344e437f5f9 I0419 18:57:38.617482 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 42.177627ms to process 1 records I0419 18:57:38.617491 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 40.94366ms to process 0 records I0419 18:57:38.617507 1 tasks_processing.go:74] worker 38 stopped. I0419 18:57:38.617572 1 recorder.go:75] Recording config/featuregate with fingerprint=c85b2fd7c44378ae57321da2e5c3b42bc5df461657504ecd1a6bd7071a014c67 I0419 18:57:38.617583 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 43.346386ms to process 1 records I0419 18:57:38.617583 1 tasks_processing.go:74] worker 49 stopped. I0419 18:57:38.617809 1 tasks_processing.go:74] worker 28 stopped. I0419 18:57:38.618991 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=7a67c516d266a2f80c5c5ac4651e6d6eaad63991cdbf123c39b0a2acf3c6d1f7 I0419 18:57:38.619065 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 42.323159ms to process 1 records I0419 18:57:38.621019 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0419 18:57:38.621045 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0419 18:57:38.621068 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0419 18:57:38.624558 1 base_controller.go:82] Caches are synced for ConfigController I0419 18:57:38.624570 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0419 18:57:38.631957 1 tasks_processing.go:74] worker 51 stopped. I0419 18:57:38.631970 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 55.197005ms to process 0 records I0419 18:57:38.632433 1 tasks_processing.go:74] worker 50 stopped. I0419 18:57:38.632536 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=f458c7ae188337a3c79ab52e76c1c576702ca12cfe19791c68bb5ff0ddff8905 I0419 18:57:38.632553 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 55.641652ms to process 1 records I0419 18:57:38.632560 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 56.722184ms to process 0 records E0419 18:57:38.632565 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) I0419 18:57:38.632578 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0419 18:57:38.632586 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 59.989135ms to process 1 records I0419 18:57:38.632595 1 tasks_processing.go:74] worker 24 stopped. I0419 18:57:38.632601 1 tasks_processing.go:74] worker 29 stopped. I0419 18:57:38.632604 1 gather_logs.go:145] no pods in namespace were found I0419 18:57:38.632611 1 tasks_processing.go:74] worker 22 stopped. I0419 18:57:38.632618 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 56.612501ms to process 0 records I0419 18:57:38.632624 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 56.034304ms to process 0 records I0419 18:57:38.632629 1 tasks_processing.go:74] worker 25 stopped. I0419 18:57:38.633247 1 tasks_processing.go:74] worker 52 stopped. I0419 18:57:38.633262 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 57.714812ms to process 0 records I0419 18:57:38.633306 1 tasks_processing.go:74] worker 31 stopped. I0419 18:57:38.633371 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=8608b45f19b1b07ce88749d409f1f14c71e5da745ea925544248a2a2516cb849 I0419 18:57:38.633382 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 58.030235ms to process 1 records I0419 18:57:38.633456 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=67672b16137befb30b89c29476649f2a517f4de7c67a026412f64cae58b8856f I0419 18:57:38.633468 1 tasks_processing.go:74] worker 18 stopped. I0419 18:57:38.633479 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=ee666b342ad3116f40ed7e393eb2ac1c07af84c0978d3591aec52b25b41869cb I0419 18:57:38.633509 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=b061ae652db0bd95b53dd9af60d88264709ca88fe837d4ecebee2e85eb4a3a3a I0419 18:57:38.633519 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 59.625265ms to process 3 records I0419 18:57:38.633612 1 tasks_processing.go:74] worker 53 stopped. I0419 18:57:38.633710 1 recorder.go:75] Recording config/oauth with fingerprint=516c2aadf4b4c505991ca68c8a15eed0a3c5cfe9e3a23a00b207bf4be9f60b3b I0419 18:57:38.633724 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 56.670063ms to process 1 records I0419 18:57:38.633732 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 58.186893ms to process 0 records I0419 18:57:38.633737 1 tasks_processing.go:74] worker 20 stopped. I0419 18:57:38.633766 1 tasks_processing.go:74] worker 14 stopped. I0419 18:57:38.633932 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=daa5b410b528e6338ebcef1d7a51bfa8bd19cc0b1fb9bd38b00afe2a233dae98 I0419 18:57:38.633962 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=e8f8c31040260b75521c26b9a4bee4c0840672414c26fa829f867a602085d041 I0419 18:57:38.633984 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=607320c83577d08c971c0574e8f8d06e853bcb6113f700793b09795f0c0eb63c I0419 18:57:38.633991 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 58.080078ms to process 3 records I0419 18:57:38.635400 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0419 18:57:38.635449 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0419 18:57:38.635528 1 operator.go:328] started I0419 18:57:38.635545 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0419 18:57:38.636576 1 tasks_processing.go:74] worker 58 stopped. I0419 18:57:38.636793 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=627531c40e04b10109f460590ad8b08ba765e5a3fda449d101e11e08cd10dbe3 I0419 18:57:38.636941 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=8bda05f4230d86cf76b4c8b4430c9669b3c01df0bddd5dfc6f8d0aacb0e3c881 I0419 18:57:38.636987 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=2e89ed1fb27829de88a7b1413777f00be94fe00f799e89bab412a287b3fefa21 I0419 18:57:38.637029 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=f7378d5d7c2036f268f284d1dc684bd64085a6e51552cbcc2d3e400bfaaf365d I0419 18:57:38.637075 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=e9d15a9b873dc015d2c1a925ffefa0f84b733891ff0c666c90d43bb78835f59c I0419 18:57:38.637114 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=38561a687050798138ebd637bcf4069b09efb2ad84a2bb7550bc4df30d264434 I0419 18:57:38.637161 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=4e064d7f921e7b4994990fe9d18b3bc5dc188916991a292e575636f670aaeee1 I0419 18:57:38.637359 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=c169c1c8207b4374e9dc97e17e0021991bf1975dd0b15237fa0983f97471e3a0 I0419 18:57:38.637421 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=ab13aa534cb3540187725c5817d06a90d8bea446d3dc0c34b9335bed69ea91e1 I0419 18:57:38.637483 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=3487579a26d8d835a53d936fa275ffe85953774c7b179eb5d05ac1c7c5b3633e I0419 18:57:38.637523 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=351e9e81e64f66d62ea8684368b065d426ce0ec620eff7d8b3bd71996d4357eb I0419 18:57:38.637552 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 62.972864ms to process 11 records I0419 18:57:38.639217 1 tasks_processing.go:74] worker 16 stopped. I0419 18:57:38.640100 1 recorder.go:75] Recording config/ingress with fingerprint=8b85d02fa49aa154a2c5cfd38cfece96e9dfa9b626e208c0ec9be4d55c940787 I0419 18:57:38.640124 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 62.335437ms to process 1 records I0419 18:57:38.640176 1 tasks_processing.go:74] worker 45 stopped. W0419 18:57:38.640425 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0419 18:57:38.640697 1 recorder.go:75] Recording config/node/ip-10-0-0-199.ec2.internal with fingerprint=60a29b0c0c202ca4f0e203f9a700ae9a72289564893a704138bec1991614a737 I0419 18:57:38.640802 1 recorder.go:75] Recording config/node/ip-10-0-1-39.ec2.internal with fingerprint=262c77b5ef91aada49443190df35d7fb83380aca1a6d5a646e7e33c27168bfcd I0419 18:57:38.640897 1 recorder.go:75] Recording config/node/ip-10-0-2-75.ec2.internal with fingerprint=24e7cd4891a87cb61b4a8dfb58ce7f887c076bb4e198e3801fa3422726189b39 I0419 18:57:38.640911 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 64.832798ms to process 3 records I0419 18:57:38.640933 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0419 18:57:38.640943 1 tasks_processing.go:74] worker 30 stopped. I0419 18:57:38.640944 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 64.154638ms to process 1 records I0419 18:57:38.640953 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 68.521536ms to process 0 records I0419 18:57:38.640961 1 tasks_processing.go:74] worker 19 stopped. I0419 18:57:38.644984 1 configmapobserver.go:84] configmaps "insights-config" not found I0419 18:57:38.645279 1 tasks_processing.go:74] worker 1 stopped. I0419 18:57:38.645477 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=3f6a22fd025c6a5a4e71789d9ec633ba86f7d5243ceb548e9b31397217653a90 I0419 18:57:38.645562 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=2b5b754bbb86bc344df0dfa81953580fc04824a6493d0044aea40e10a84d6e4f I0419 18:57:38.645571 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 71.650664ms to process 2 records I0419 18:57:38.645582 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 72.857431ms to process 0 records I0419 18:57:38.645588 1 tasks_processing.go:74] worker 8 stopped. I0419 18:57:38.649975 1 controller.go:254] Source scaController *sca.Controller is not ready I0419 18:57:38.649989 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0419 18:57:38.649993 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0419 18:57:38.649998 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0419 18:57:38.650002 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0419 18:57:38.650022 1 controller.go:531] The operator is still being initialized I0419 18:57:38.650030 1 controller.go:554] The operator is healthy I0419 18:57:38.656773 1 tasks_processing.go:74] worker 42 stopped. I0419 18:57:38.657066 1 recorder.go:75] Recording config/version with fingerprint=c66b7dd1b2c614e244bc8b2e2a2661fbe492922f41fb188dfcee78be0ff2c647 I0419 18:57:38.657086 1 recorder.go:75] Recording config/id with fingerprint=7332d470268ca48d89bcf3aa33a16bd6a65d5f9cfecdc6eed98747b9a33f01ff I0419 18:57:38.657099 1 gather.go:177] gatherer "clusterconfig" function "version" took 80.867043ms to process 2 records I0419 18:57:38.659058 1 tasks_processing.go:74] worker 21 stopped. I0419 18:57:38.660422 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-m26ts with fingerprint=d5f06a1f3bf0b57bc7ae0e320a80da6b8afdc6fa4d6d347944fc2437394f139a I0419 18:57:38.660470 1 recorder.go:75] Recording config/running_containers with fingerprint=88b1204a29083e5d10fb4eb7cd7d49fb0e6881c197629b0bbdecd81b3c388fca I0419 18:57:38.660480 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 83.821835ms to process 2 records I0419 18:57:38.662855 1 prometheus_rules.go:88] Prometheus rules successfully created I0419 18:57:38.663114 1 tasks_processing.go:74] worker 41 stopped. I0419 18:57:38.663562 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=44db9464a044091dc553029d677a5c1a5b484126dc139f65ac31f7b0c3317739 I0419 18:57:38.663766 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=1c77201cf0fcbfd17db514847b5a201e90beaeea906797d2daf73ce7d65e78c2 I0419 18:57:38.663776 1 gather.go:177] gatherer "clusterconfig" function "crds" took 87.182004ms to process 2 records E0419 18:57:38.664118 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27f6967457-6f69-44b3-af20-078d4a26e1e4%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:43985->172.30.0.10:53: read: connection refused I0419 18:57:38.664134 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27f6967457-6f69-44b3-af20-078d4a26e1e4%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:43985->172.30.0.10:53: read: connection refused I0419 18:57:38.669647 1 tasks_processing.go:74] worker 36 stopped. I0419 18:57:38.669663 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 93.812117ms to process 0 records I0419 18:57:38.709681 1 tasks_processing.go:74] worker 37 stopped. E0419 18:57:38.709703 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0419 18:57:38.709712 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0419 18:57:38.709717 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0419 18:57:38.709733 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=ad286723f58bdcfc37aeba3ec5b4110c08e1af59cd34d14b4bfaab02d18e4856 I0419 18:57:38.709782 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0419 18:57:38.709793 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=2ab5ab7b1b10d7fcf1197bb24dea7c90f400e4effc18c7356873209d54fdf84b I0419 18:57:38.709797 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0419 18:57:38.709801 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0419 18:57:38.709847 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0419 18:57:38.709856 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0419 18:57:38.709861 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 133.482269ms to process 7 records I0419 18:57:38.736619 1 base_controller.go:82] Caches are synced for LoggingSyncer I0419 18:57:38.736631 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0419 18:57:38.740815 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0419 18:57:38.745976 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found W0419 18:57:38.748816 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:33415->172.30.0.10:53: read: connection refused I0419 18:57:38.748830 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.12:33415->172.30.0.10:53: read: connection refused I0419 18:57:38.774601 1 tasks_processing.go:74] worker 59 stopped. E0419 18:57:38.774620 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0419 18:57:38.774627 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2ppd8c54leko56t30id91u7ej78p9pm4-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2ppd8c54leko56t30id91u7ej78p9pm4-primary-cert-bundle-secret" not found I0419 18:57:38.774700 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=90548ad8383aa0f9e4ee9b76123be40ecde9c60300d1749a1881e877a2ed92af I0419 18:57:38.774716 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 198.462703ms to process 1 records I0419 18:57:39.046989 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0419 18:57:39.047009 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0419 18:57:39.047954 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-drmnn pod in namespace openshift-dns (previous: false). I0419 18:57:39.275464 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-drmnn pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-drmnn\" is waiting to start: ContainerCreating" I0419 18:57:39.275484 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-drmnn\" is waiting to start: ContainerCreating" I0419 18:57:39.275493 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-drmnn pod in namespace openshift-dns (previous: false). I0419 18:57:39.452044 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-drmnn pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-drmnn\" is waiting to start: ContainerCreating" I0419 18:57:39.452061 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-drmnn\" is waiting to start: ContainerCreating" I0419 18:57:39.452073 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-w9846 pod in namespace openshift-dns (previous: false). W0419 18:57:39.637288 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0419 18:57:39.685029 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-w9846 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-w9846\" is waiting to start: ContainerCreating" I0419 18:57:39.685049 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-w9846\" is waiting to start: ContainerCreating" I0419 18:57:39.685056 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-w9846 pod in namespace openshift-dns (previous: false). I0419 18:57:39.851493 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-w9846 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-w9846\" is waiting to start: ContainerCreating" I0419 18:57:39.851516 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-w9846\" is waiting to start: ContainerCreating" I0419 18:57:39.851528 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-wmz8v pod in namespace openshift-dns (previous: false). I0419 18:57:40.054079 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0419 18:57:40.072538 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wmz8v pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-wmz8v\" is waiting to start: ContainerCreating" I0419 18:57:40.072552 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-wmz8v\" is waiting to start: ContainerCreating" I0419 18:57:40.072561 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-wmz8v pod in namespace openshift-dns (previous: false). I0419 18:57:40.258891 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-wmz8v pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-wmz8v\" is waiting to start: ContainerCreating" I0419 18:57:40.258907 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-wmz8v\" is waiting to start: ContainerCreating" I0419 18:57:40.258918 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-8sgj5 pod in namespace openshift-dns (previous: false). I0419 18:57:40.452182 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0419 18:57:40.452201 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-hz7qq pod in namespace openshift-dns (previous: false). I0419 18:57:40.457426 1 tasks_processing.go:74] worker 62 stopped. I0419 18:57:40.457476 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=4e20016a5690d9f119886243a7b28b4868fef5255408d46b90c5b58dc6742cae I0419 18:57:40.457513 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=cda1376921ebe5ce73debbc345eaea19a2af381752088f6a76fcf78f19a33034 I0419 18:57:40.457542 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0419 18:57:40.457566 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=0fa7bd9fdfe9d7f01b5460c00851e0712ba5871eb426be9d6edacaa3817fbb54 I0419 18:57:40.457583 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0419 18:57:40.457623 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=54f75f3d84544c3e85e9f8ebff54acdf2725f1c74ab9b5f3b2620540e7f8289d I0419 18:57:40.457654 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=bc6affd37e608b69f27df01567831e0e78a269f67be535cdafce3069b87785cc I0419 18:57:40.457677 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=4b1b9931b2a5309ea118287f32fc58be240a258eabe84b2537d15352319de6f0 I0419 18:57:40.457691 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=16ea00db1fce7bb7132ec6b83f9d2cf9ffccd30bdfc2b78356a991116972470f I0419 18:57:40.457707 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=b6e06eaccc21efeba68ab96205d99c887c0747df80236c5a8f5552800966e39f I0419 18:57:40.457719 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0419 18:57:40.457735 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=aa09f8ad997110b46114c6063f7ab2c6abbd1d631ff838da109f46493314f7dd I0419 18:57:40.457745 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0419 18:57:40.457760 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=2e82f1aae968922a6e6beada7d6faa436fd29a7b4af599ce74d2f20af71e84b8 I0419 18:57:40.457768 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0419 18:57:40.457786 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=cb32071589660fc2fa753482cd6371dae0c4885fa32ada55d7f1fb73ad1b4fff I0419 18:57:40.457796 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0419 18:57:40.457811 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=bdae4646211142ca1b0a9cf255a1dce10f5597119eec8a63df0ff41be3bf5d1b I0419 18:57:40.457937 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=20a1be575bc16aabd31b0f4fe367c9480fe38fff0fb9e2b6ba84e4088ee24692 I0419 18:57:40.457948 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0419 18:57:40.457955 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0419 18:57:40.457975 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0419 18:57:40.457998 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=fb1ba95bdb519ef6ce92848d3e62961de783a3f34a2d24631dfad30783c8bf01 I0419 18:57:40.458021 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=28fd3dee84ee351174c08204df4fad10977514004f4e808828e3a2aeffcd6669 I0419 18:57:40.458038 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0419 18:57:40.458054 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=a3c6acecbe5a05eced95260ca63f1c680bbd8f34e16af370e834c04d08944140 I0419 18:57:40.458063 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0419 18:57:40.458077 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=6d2ee71fd89104f57aceb35e297ebc1b22283b443b9ec5811da05a4cbb95f97c I0419 18:57:40.458092 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=f64e6c1f9f8e278e43d6605b46888c63ba903945ac11b2b4a08880ac035d1d79 I0419 18:57:40.458105 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=90d4b746f3e908070b0bd9a94cf1e8a5bd96a635c5bd8c13b342deb3fb4cb8f0 I0419 18:57:40.458118 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=1479c35bda118a773833c9fad7c8cbef8a4ceddaee5f33f50f9f34ce361d5981 I0419 18:57:40.458130 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=603982a9763cfc5bebc43f69d478c65f03db46ff203c7dff324b68e98b43252c I0419 18:57:40.458154 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=83cd3458317beae63a366309bbe94514d11b6671349271d2c7c58719be197de9 I0419 18:57:40.458169 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0419 18:57:40.458176 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0419 18:57:40.458183 1 gather.go:177] gatherer "clusterconfig" function "operators" took 1.883699114s to process 35 records W0419 18:57:40.636654 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0419 18:57:40.651793 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0419 18:57:40.651809 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-xkjd2 pod in namespace openshift-dns (previous: false). I0419 18:57:40.852376 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0419 18:57:40.852395 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7697c6557-625sz pod in namespace openshift-image-registry (previous: false). I0419 18:57:41.053537 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7697c6557-625sz pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7697c6557-625sz\" is waiting to start: ContainerCreating" I0419 18:57:41.053561 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7697c6557-625sz\" is waiting to start: ContainerCreating" I0419 18:57:41.053571 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7697c6557-qwwrv pod in namespace openshift-image-registry (previous: false). I0419 18:57:41.251605 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7697c6557-qwwrv pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7697c6557-qwwrv\" is waiting to start: ContainerCreating" I0419 18:57:41.251620 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7697c6557-qwwrv\" is waiting to start: ContainerCreating" I0419 18:57:41.251630 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-bc8fd586b-jj2br pod in namespace openshift-image-registry (previous: false). I0419 18:57:41.456615 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-bc8fd586b-jj2br pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-bc8fd586b-jj2br\" is waiting to start: ContainerCreating" I0419 18:57:41.456631 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-bc8fd586b-jj2br\" is waiting to start: ContainerCreating" I0419 18:57:41.456640 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-6qfvt pod in namespace openshift-image-registry (previous: false). W0419 18:57:41.637063 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0419 18:57:41.652378 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0419 18:57:41.652392 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-h2hpb pod in namespace openshift-image-registry (previous: false). I0419 18:57:41.853274 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0419 18:57:41.853293 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-lbkz4 pod in namespace openshift-image-registry (previous: false). I0419 18:57:42.053642 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0419 18:57:42.053660 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5bf7d97bcf-htd5w pod in namespace openshift-ingress (previous: false). I0419 18:57:42.253186 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5bf7d97bcf-htd5w pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5bf7d97bcf-htd5w\" is waiting to start: ContainerCreating" I0419 18:57:42.253205 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5bf7d97bcf-htd5w\" is waiting to start: ContainerCreating" I0419 18:57:42.253217 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5f94d75645-s62m6 pod in namespace openshift-ingress (previous: false). I0419 18:57:42.451779 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5f94d75645-s62m6 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5f94d75645-s62m6\" is waiting to start: ContainerCreating" I0419 18:57:42.451796 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5f94d75645-s62m6\" is waiting to start: ContainerCreating" I0419 18:57:42.451806 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-5f94d75645-wxmx6 pod in namespace openshift-ingress (previous: false). W0419 18:57:42.636910 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0419 18:57:42.652552 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-5f94d75645-wxmx6 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-5f94d75645-wxmx6\" is waiting to start: ContainerCreating" I0419 18:57:42.652568 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-5f94d75645-wxmx6\" is waiting to start: ContainerCreating" I0419 18:57:42.652578 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-98lr7 pod in namespace openshift-ingress-canary (previous: false). I0419 18:57:42.851707 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-n5svd pod in namespace openshift-ingress-canary (previous: false). I0419 18:57:43.053679 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-phbfp pod in namespace openshift-ingress-canary (previous: false). I0419 18:57:43.253140 1 tasks_processing.go:74] worker 63 stopped. I0419 18:57:43.253230 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=6f05a8b13fa2145d9a88ef4190f739a586600dfcf0e9e2bb889edbd1314e3025 I0419 18:57:43.253282 1 recorder.go:75] Recording events/openshift-dns with fingerprint=6220801ad980d2a797eedfd3b3d202ca5f5c62b74fddd3b4cf2c358ab0bebe28 I0419 18:57:43.253372 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=fff7a1623fecf8b0ecd4d565da16ec54a9049f5b00ef994ecc321da5f5dc4a45 I0419 18:57:43.253401 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=d2061feadb55738025227b51d8ecae2aea3aec4dde02e8df798c2ed3c251bb6f I0419 18:57:43.253446 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=c0075672321cbe0ee2c7131d3babb5699cdfd502828ceffcd17c95284756ae8e I0419 18:57:43.253483 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=05df8350bf4243cf38f9f78fc74b22d871a1fe29c21ef44a26347ea6b2fb9d75 I0419 18:57:43.253491 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-98lr7/serve-healthcheck-canary_current.log with fingerprint=e885de66f3faca2f91259ec7433de7f5aaacec5042523fd57a06bd854d82539e I0419 18:57:43.253495 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-n5svd/serve-healthcheck-canary_current.log with fingerprint=9fe43f8c99cb2020f9cb3c544f493624b8123cc70453467d0fc0eb615729a73c I0419 18:57:43.253499 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-phbfp/serve-healthcheck-canary_current.log with fingerprint=0dba187abf74c83dd2987b0da950df6b940fe948e9d0554eef795149f18407aa I0419 18:57:43.253505 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.676972966s to process 9 records W0419 18:57:43.636691 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0419 18:57:43.636717 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0419 18:57:43.636733 1 tasks_processing.go:74] worker 26 stopped. E0419 18:57:43.636744 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0419 18:57:43.636754 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0419 18:57:43.636769 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0419 18:57:43.636778 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.062573354s to process 1 records I0419 18:57:51.247783 1 tasks_processing.go:74] worker 55 stopped. I0419 18:57:51.247818 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0419 18:57:51.247831 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.67232328s to process 1 records I0419 18:57:51.487877 1 configmapobserver.go:84] configmaps "insights-config" not found I0419 18:57:51.986599 1 tasks_processing.go:74] worker 17 stopped. I0419 18:57:51.986951 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=7ddf795073bc5395a95429a28409923bc71b328da94f586bee093d6f578708e6 I0419 18:57:51.986970 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.410349816s to process 1 records E0419 18:57:51.987026 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.415s with: function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_configs\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0419 18:57:51.988130 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_configs" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0419 18:57:51.988145 1 periodic.go:216] Running workloads gatherer I0419 18:57:51.988160 1 tasks_processing.go:45] number of workers: 2 I0419 18:57:51.988167 1 tasks_processing.go:69] worker 1 listening for tasks. I0419 18:57:51.988171 1 tasks_processing.go:71] worker 1 working on workload_info task. I0419 18:57:51.988179 1 tasks_processing.go:69] worker 0 listening for tasks. I0419 18:57:51.988198 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0419 18:57:52.010967 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0419 18:57:52.015570 1 tasks_processing.go:74] worker 0 stopped. I0419 18:57:52.015584 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 27.353731ms to process 0 records I0419 18:57:52.018702 1 gather_workloads_info.go:387] No image sha256:730d1b6988025bef0daa3a9a5d8467ec4a26b0382cc52f91c3375b4590d3518a (8ms) I0419 18:57:52.029579 1 gather_workloads_info.go:387] No image sha256:a043239802b3eb8b323d285193d2527fad0ecec98ca91d188a3472a2fac8ae04 (11ms) I0419 18:57:52.038515 1 gather_workloads_info.go:387] No image sha256:5b6212b8f539f08e78417d8a4b7485ca0b4e7927cacd7b752742a28841bc8ccd (9ms) I0419 18:57:52.049037 1 gather_workloads_info.go:387] No image sha256:89277d8d4560d71db88c2dcc67c992a24544ca21810920b609c1d49d53b4a287 (11ms) I0419 18:57:52.057169 1 gather_workloads_info.go:387] No image sha256:875c77e5d144f03fb91d8cee0259f6966683ca88d1bf818dbf4652c16b70312c (8ms) I0419 18:57:52.064061 1 gather_workloads_info.go:387] No image sha256:0a4dfb8d4c1b3849319d45b4c54dff26a7238a2c08fcaa121f93073e95ab12e8 (7ms) I0419 18:57:52.071747 1 gather_workloads_info.go:387] No image sha256:30e597ec5d6bb96ff70a4f8688c748b659cd4fd5d73d222e8701821d236795c5 (8ms) I0419 18:57:52.079368 1 gather_workloads_info.go:387] No image sha256:6814f5fced0ef219d06374011c68a11a3da788a764a00a69ade435466d9ee240 (8ms) I0419 18:57:52.091569 1 gather_workloads_info.go:387] No image sha256:e1ba458cf6f0b3606c90880da72db8ab99cd11040bae84baebe3ff2e0d1ea075 (12ms) I0419 18:57:52.105681 1 gather_workloads_info.go:387] No image sha256:934d8e8c50f3c609b8eea80d1051111fe3d066fe8c65c79572072ae55fcb0a86 (14ms) I0419 18:57:52.118634 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (13ms) I0419 18:57:52.220380 1 gather_workloads_info.go:387] No image sha256:637d41f067a5239096fc22b135181cda5113da833f1370e7a73965e83792e93a (102ms) I0419 18:57:52.319404 1 gather_workloads_info.go:387] No image sha256:084aa9b0f8a6d478549dc384d4e66da13ee9b25cc98531da861cc19dee2a9e8f (99ms) I0419 18:57:52.420379 1 gather_workloads_info.go:387] No image sha256:000105ef5150e7079b90a613fb9e6193e2a6ef9b1908d2dce44f2395d4fd070f (101ms) I0419 18:57:52.520021 1 gather_workloads_info.go:387] No image sha256:90a8ffd9643ebb16a6a8c04bb38cf9ed58903e9d3bf836c68f399193db5edaf6 (100ms) I0419 18:57:52.620035 1 gather_workloads_info.go:387] No image sha256:3bba1358d4a0ae878ff491c0c2cbfffe60649e110b40342b878fe8fa332f8858 (100ms) I0419 18:57:52.723519 1 gather_workloads_info.go:387] No image sha256:d64bea34bf3e1bb0b3a701c3ff14e66665afc1b050f28124ad7e6888eaec3a81 (103ms) I0419 18:57:52.818852 1 gather_workloads_info.go:387] No image sha256:ab60623bb32f7e75fca71ef65137731cae347a21c7a4091dfd583fa00732721c (95ms) I0419 18:57:52.919978 1 gather_workloads_info.go:387] No image sha256:ce138e8cf5b96557b1864ef6f27c2608bdca59be2611804366cef7169c36291e (101ms) I0419 18:57:53.019276 1 gather_workloads_info.go:387] No image sha256:5c6d21c3f97366bc7ab57031cc027b67405a684bd804ce364ed5998b0685eaca (99ms) I0419 18:57:53.118941 1 gather_workloads_info.go:387] No image sha256:08c5a78c8a5af04c549e2273aaf4bb452a75bf038d68aa9d01bb2aff66c30e90 (100ms) I0419 18:57:53.219745 1 gather_workloads_info.go:387] No image sha256:673ebc8cc22c56c8f410e011b2fa950d28cf7b6420e17fdb6580d6cb10523384 (101ms) I0419 18:57:53.322228 1 gather_workloads_info.go:387] No image sha256:5808401268394502d335281ea1a294b07210461b986b58f91d7d1f29c0029c6d (102ms) I0419 18:57:53.418914 1 gather_workloads_info.go:387] No image sha256:5ac9c549d65fc1d8bc900773bebee43e9192bcec1bb5fa46afb4597230c16ac7 (97ms) I0419 18:57:53.519184 1 gather_workloads_info.go:387] No image sha256:25c148fd380b1a9db3f6039d2e0eabc489a954921452391390ba9192b2325678 (100ms) I0419 18:57:53.619189 1 gather_workloads_info.go:387] No image sha256:48e883a3932aea9457f55cd4628d21397d429d4828f42f95d1c903d9d5395bde (100ms) I0419 18:57:53.719122 1 gather_workloads_info.go:387] No image sha256:55b1db6038c5beaed54c626e3343b7a8589cc0be8dc41d1a66b4deab766ff520 (100ms) I0419 18:57:53.819721 1 gather_workloads_info.go:387] No image sha256:e8b96d9318b3b8c9ed0afe4e6381f635c6b0c2f20772044ac68001ed49af2c87 (101ms) I0419 18:57:53.851088 1 configmapobserver.go:84] configmaps "insights-config" not found I0419 18:57:53.919626 1 gather_workloads_info.go:387] No image sha256:15677f0b70e6aa2dfaf088e45fc1a425c22bf6fda326b8116f87e88b6694dfab (100ms) I0419 18:57:54.019670 1 gather_workloads_info.go:387] No image sha256:4733236617781e3469ffffb15e4daaa1f14ea8e1c52b426a3787a4f1f2945424 (100ms) I0419 18:57:54.050489 1 configmapobserver.go:84] configmaps "insights-config" not found I0419 18:57:54.119290 1 gather_workloads_info.go:387] No image sha256:5aaea0419169e55832cc27acfe0fe3b9513a343d6bdf71d3da1575ed322245d0 (100ms) I0419 18:57:54.219706 1 gather_workloads_info.go:387] No image sha256:e3732e356ae2324565c74cae57d5d016917314fa293ddaa3a68ee9ae030c6f07 (100ms) I0419 18:57:54.319292 1 gather_workloads_info.go:387] No image sha256:d6fbe0075cbb12bfd287c973704fadf97154c7f73e370733d976a40835e9155a (100ms) I0419 18:57:54.319322 1 tasks_processing.go:74] worker 1 stopped. E0419 18:57:54.319332 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0419 18:57:54.319580 1 recorder.go:75] Recording config/workload_info with fingerprint=4c74d426acb3d4314d624f97ff950e39e1131c9d037fcd19f8f2428449c3cdcc I0419 18:57:54.319595 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.331144389s to process 1 records E0419 18:57:54.319623 1 periodic.go:254] "Unhandled Error" err="workloads failed after 2.331s with: function \"workload_info\" failed with an error" I0419 18:57:54.320728 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0419 18:57:54.320741 1 periodic.go:216] Running conditional gatherer I0419 18:57:54.326806 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules I0419 18:57:54.333543 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.12:51347->172.30.0.10:53: read: connection refused E0419 18:57:54.333847 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0419 18:57:54.333921 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0419 18:57:54.339355 1 conditional_gatherer.go:392] cluster version is '4.21.9' E0419 18:57:54.339367 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339372 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339375 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339379 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339382 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339384 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339387 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339390 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0419 18:57:54.339393 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0419 18:57:54.339407 1 tasks_processing.go:45] number of workers: 3 I0419 18:57:54.339414 1 tasks_processing.go:69] worker 2 listening for tasks. I0419 18:57:54.339419 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0419 18:57:54.339482 1 tasks_processing.go:69] worker 0 listening for tasks. I0419 18:57:54.339494 1 tasks_processing.go:69] worker 1 listening for tasks. I0419 18:57:54.339557 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0419 18:57:54.339566 1 tasks_processing.go:74] worker 1 stopped. I0419 18:57:54.339578 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0419 18:57:54.339652 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0419 18:57:54.339665 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.205µs to process 1 records I0419 18:57:54.339694 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0419 18:57:54.339701 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.823µs to process 1 records I0419 18:57:54.339706 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 221.577µs to process 0 records I0419 18:57:54.339711 1 tasks_processing.go:74] worker 2 stopped. I0419 18:57:54.339714 1 tasks_processing.go:74] worker 0 stopped. I0419 18:57:54.339730 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.9/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.12:51347->172.30.0.10:53: read: connection refused I0419 18:57:54.339744 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0419 18:57:54.365902 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=e1e88766f5a36f42ac2b68b1e324d97657069a4eab5e704e0b36960f91878d4a I0419 18:57:54.366027 1 diskrecorder.go:70] Writing 103 records to /var/lib/insights-operator/insights-2026-04-19-185754.tar.gz I0419 18:57:54.372150 1 diskrecorder.go:51] Wrote 103 records to disk in 6ms I0419 18:57:54.372178 1 periodic.go:285] Gathering cluster info every 2h0m0s I0419 18:57:54.372192 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0419 18:58:07.156197 1 configmapobserver.go:84] configmaps "insights-config" not found I0419 18:59:03.237447 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="06fe76fa0ba6b517f039980117327694065e6d8711f7ef998a60ab2101a7ef2d") W0419 18:59:03.237480 1 builder.go:160] Restart triggered because of file /var/run/configmaps/service-ca-bundle/service-ca.crt was created I0419 18:59:03.237537 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0419 18:59:03.237546 1 base_controller.go:181] Shutting down ConfigController ... I0419 18:59:03.237562 1 base_controller.go:181] Shutting down LoggingSyncer ... I0419 18:59:03.237592 1 periodic.go:177] Shutting down I0419 18:59:03.237602 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0419 18:59:03.237608 1 base_controller.go:113] All LoggingSyncer workers have been terminated I0419 18:59:03.237546 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="f89e583c6f9709ca0fefba2415ffbad294a4a3eda8f1ab98f804e8c5c17b28d6") I0419 18:59:03.237642 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" E0419 18:59:03.237653 1 controller.go:341] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled I0419 18:59:03.237651 1 base_controller.go:123] Shutting down worker of ConfigController controller ... E0419 18:59:03.237683 1 controller.go:341] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled