W0326 23:15:36.946430 1 cmd.go:257] Using insecure, self-signed certificates I0326 23:15:37.366884 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0326 23:15:37.367196 1 observer_polling.go:159] Starting file observer I0326 23:15:37.758425 1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$ I0326 23:15:37.758640 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0326 23:15:37.759166 1 secure_serving.go:57] Forcing use of http/1.1 only W0326 23:15:37.759187 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0326 23:15:37.759191 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0326 23:15:37.759195 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0326 23:15:37.759198 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0326 23:15:37.759200 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0326 23:15:37.759203 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0326 23:15:37.759242 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0326 23:15:37.762667 1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryption KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot] I0326 23:15:37.762706 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"e2336d2e-30f6-4a41-9630-c8112d6dddac", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryption", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}} I0326 23:15:37.763362 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0326 23:15:37.763385 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0326 23:15:37.763375 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0326 23:15:37.763379 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0326 23:15:37.763414 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0326 23:15:37.763420 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0326 23:15:37.763716 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1287528898/tls.crt::/tmp/serving-cert-1287528898/tls.key" I0326 23:15:37.764148 1 secure_serving.go:213] Serving securely on [::]:8443 I0326 23:15:37.764175 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0326 23:15:37.766641 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0326 23:15:37.766668 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0326 23:15:37.766780 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0326 23:15:37.771185 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0326 23:15:37.771200 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0326 23:15:37.775190 1 secretconfigobserver.go:119] support secret does not exist I0326 23:15:37.780176 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0326 23:15:37.786265 1 secretconfigobserver.go:119] support secret does not exist I0326 23:15:37.788826 1 recorder.go:176] Pruning old reports every 7h16m50s, max age is 288h0m0s I0326 23:15:37.792861 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0326 23:15:37.792866 1 periodic.go:216] Running clusterconfig gatherer I0326 23:15:37.792879 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0326 23:15:37.792906 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0326 23:15:37.792907 1 tasks_processing.go:45] number of workers: 64 I0326 23:15:37.792913 1 insightsreport.go:296] Starting report retriever I0326 23:15:37.792920 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0326 23:15:37.792932 1 tasks_processing.go:69] worker 1 listening for tasks. I0326 23:15:37.792934 1 tasks_processing.go:69] worker 2 listening for tasks. I0326 23:15:37.792932 1 tasks_processing.go:69] worker 4 listening for tasks. I0326 23:15:37.792941 1 tasks_processing.go:69] worker 3 listening for tasks. I0326 23:15:37.792945 1 tasks_processing.go:69] worker 0 listening for tasks. I0326 23:15:37.792950 1 tasks_processing.go:71] worker 4 working on storage_cluster task. I0326 23:15:37.792952 1 tasks_processing.go:69] worker 15 listening for tasks. I0326 23:15:37.792955 1 tasks_processing.go:69] worker 46 listening for tasks. I0326 23:15:37.792956 1 tasks_processing.go:71] worker 0 working on olm_operators task. I0326 23:15:37.792960 1 tasks_processing.go:71] worker 15 working on nodenetworkconfigurationpolicies task. I0326 23:15:37.792961 1 tasks_processing.go:69] worker 26 listening for tasks. I0326 23:15:37.792966 1 tasks_processing.go:69] worker 28 listening for tasks. I0326 23:15:37.792971 1 tasks_processing.go:69] worker 29 listening for tasks. I0326 23:15:37.792976 1 tasks_processing.go:69] worker 30 listening for tasks. I0326 23:15:37.792969 1 tasks_processing.go:69] worker 20 listening for tasks. I0326 23:15:37.792980 1 tasks_processing.go:69] worker 16 listening for tasks. I0326 23:15:37.792984 1 tasks_processing.go:69] worker 21 listening for tasks. I0326 23:15:37.792985 1 tasks_processing.go:69] worker 17 listening for tasks. I0326 23:15:37.792989 1 tasks_processing.go:69] worker 31 listening for tasks. I0326 23:15:37.792990 1 tasks_processing.go:69] worker 19 listening for tasks. I0326 23:15:37.792994 1 tasks_processing.go:69] worker 32 listening for tasks. I0326 23:15:37.792986 1 tasks_processing.go:69] worker 10 listening for tasks. I0326 23:15:37.792998 1 tasks_processing.go:69] worker 23 listening for tasks. I0326 23:15:37.792998 1 tasks_processing.go:69] worker 5 listening for tasks. I0326 23:15:37.793004 1 tasks_processing.go:69] worker 34 listening for tasks. I0326 23:15:37.792994 1 tasks_processing.go:69] worker 22 listening for tasks. I0326 23:15:37.793000 1 tasks_processing.go:69] worker 33 listening for tasks. I0326 23:15:37.793011 1 tasks_processing.go:69] worker 25 listening for tasks. I0326 23:15:37.793006 1 tasks_processing.go:69] worker 18 listening for tasks. I0326 23:15:37.793017 1 tasks_processing.go:69] worker 8 listening for tasks. I0326 23:15:37.793019 1 tasks_processing.go:69] worker 9 listening for tasks. I0326 23:15:37.793021 1 tasks_processing.go:69] worker 11 listening for tasks. I0326 23:15:37.793023 1 tasks_processing.go:69] worker 37 listening for tasks. I0326 23:15:37.793026 1 tasks_processing.go:69] worker 14 listening for tasks. I0326 23:15:37.793028 1 tasks_processing.go:69] worker 51 listening for tasks. I0326 23:15:37.793006 1 tasks_processing.go:69] worker 24 listening for tasks. I0326 23:15:37.793031 1 tasks_processing.go:69] worker 52 listening for tasks. I0326 23:15:37.793034 1 tasks_processing.go:69] worker 54 listening for tasks. I0326 23:15:37.793030 1 tasks_processing.go:69] worker 13 listening for tasks. I0326 23:15:37.793039 1 tasks_processing.go:69] worker 55 listening for tasks. I0326 23:15:37.793040 1 tasks_processing.go:69] worker 44 listening for tasks. I0326 23:15:37.793043 1 tasks_processing.go:69] worker 49 listening for tasks. I0326 23:15:37.793007 1 tasks_processing.go:69] worker 7 listening for tasks. I0326 23:15:37.793047 1 tasks_processing.go:69] worker 53 listening for tasks. I0326 23:15:37.793050 1 tasks_processing.go:69] worker 39 listening for tasks. I0326 23:15:37.793052 1 tasks_processing.go:69] worker 61 listening for tasks. I0326 23:15:37.793011 1 tasks_processing.go:69] worker 35 listening for tasks. I0326 23:15:37.793055 1 tasks_processing.go:71] worker 23 working on metrics task. I0326 23:15:37.793057 1 tasks_processing.go:71] worker 53 working on node_features task. I0326 23:15:37.793060 1 tasks_processing.go:69] worker 59 listening for tasks. I0326 23:15:37.793060 1 tasks_processing.go:69] worker 40 listening for tasks. I0326 23:15:37.793061 1 tasks_processing.go:69] worker 57 listening for tasks. I0326 23:15:37.793061 1 tasks_processing.go:69] worker 45 listening for tasks. I0326 23:15:37.793071 1 tasks_processing.go:69] worker 62 listening for tasks. I0326 23:15:37.793072 1 tasks_processing.go:69] worker 38 listening for tasks. I0326 23:15:37.793075 1 tasks_processing.go:71] worker 45 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0326 23:15:37.793071 1 tasks_processing.go:71] worker 57 working on image task. I0326 23:15:37.793081 1 tasks_processing.go:71] worker 30 working on machine_autoscalers task. I0326 23:15:37.793078 1 tasks_processing.go:69] worker 63 listening for tasks. I0326 23:15:37.793087 1 tasks_processing.go:71] worker 20 working on oauths task. I0326 23:15:37.793092 1 tasks_processing.go:71] worker 14 working on support_secret task. I0326 23:15:37.793094 1 tasks_processing.go:71] worker 44 working on proxies task. I0326 23:15:37.793097 1 tasks_processing.go:71] worker 22 working on nodenetworkstates task. I0326 23:15:37.793100 1 tasks_processing.go:71] worker 21 working on machine_configs task. I0326 23:15:37.793084 1 tasks_processing.go:71] worker 37 working on container_images task. I0326 23:15:37.793087 1 tasks_processing.go:71] worker 54 working on schedulers task. I0326 23:15:37.793146 1 tasks_processing.go:71] worker 24 working on openstack_controlplanes task. I0326 23:15:37.793148 1 tasks_processing.go:71] worker 34 working on crds task. I0326 23:15:37.793154 1 tasks_processing.go:71] worker 2 working on machine_sets task. I0326 23:15:37.792961 1 tasks_processing.go:69] worker 27 listening for tasks. I0326 23:15:37.793166 1 tasks_processing.go:71] worker 17 working on sap_config task. I0326 23:15:37.793092 1 tasks_processing.go:71] worker 25 working on authentication task. I0326 23:15:37.793192 1 tasks_processing.go:71] worker 49 working on aggregated_monitoring_cr_names task. I0326 23:15:37.793213 1 tasks_processing.go:71] worker 31 working on operators task. I0326 23:15:37.793262 1 tasks_processing.go:71] worker 7 working on monitoring_persistent_volumes task. I0326 23:15:37.793012 1 tasks_processing.go:69] worker 6 listening for tasks. I0326 23:15:37.793026 1 tasks_processing.go:69] worker 56 listening for tasks. I0326 23:15:37.793031 1 tasks_processing.go:69] worker 42 listening for tasks. I0326 23:15:37.793016 1 tasks_processing.go:69] worker 36 listening for tasks. I0326 23:15:37.793035 1 tasks_processing.go:69] worker 43 listening for tasks. I0326 23:15:37.793038 1 tasks_processing.go:69] worker 48 listening for tasks. I0326 23:15:37.793050 1 tasks_processing.go:69] worker 50 listening for tasks. I0326 23:15:37.793313 1 tasks_processing.go:71] worker 52 working on sap_datahubs task. I0326 23:15:37.793322 1 tasks_processing.go:71] worker 8 working on openstack_version task. I0326 23:15:37.792951 1 tasks_processing.go:71] worker 3 working on infrastructures task. I0326 23:15:37.793352 1 tasks_processing.go:71] worker 5 working on version task. I0326 23:15:37.793399 1 tasks_processing.go:71] worker 51 working on machine_config_pools task. I0326 23:15:37.793057 1 tasks_processing.go:71] worker 39 working on validating_webhook_configurations task. I0326 23:15:37.793660 1 tasks_processing.go:71] worker 18 working on lokistack task. I0326 23:15:37.793687 1 tasks_processing.go:71] worker 55 working on tsdb_status task. W0326 23:15:37.793719 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0326 23:15:37.793735 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 28.558µs to process 0 records I0326 23:15:37.793742 1 tasks_processing.go:71] worker 13 working on networks task. I0326 23:15:37.793782 1 tasks_processing.go:71] worker 9 working on cost_management_metrics_configs task. I0326 23:15:37.793794 1 tasks_processing.go:71] worker 19 working on qemu_kubevirt_launcher_logs task. I0326 23:15:37.793832 1 tasks_processing.go:71] worker 10 working on openstack_dataplanedeployments task. I0326 23:15:37.793933 1 tasks_processing.go:71] worker 33 working on mutating_webhook_configurations task. I0326 23:15:37.793057 1 tasks_processing.go:69] worker 58 listening for tasks. I0326 23:15:37.793062 1 tasks_processing.go:71] worker 46 working on install_plans task. I0326 23:15:37.793058 1 tasks_processing.go:71] worker 16 working on ceph_cluster task. I0326 23:15:37.793045 1 tasks_processing.go:69] worker 41 listening for tasks. I0326 23:15:37.794043 1 tasks_processing.go:71] worker 62 working on container_runtime_configs task. I0326 23:15:37.794047 1 tasks_processing.go:71] worker 41 working on nodes task. I0326 23:15:37.793067 1 tasks_processing.go:71] worker 26 working on openshift_machine_api_events task. I0326 23:15:37.793784 1 tasks_processing.go:71] worker 11 working on node_logs task. I0326 23:15:37.793065 1 tasks_processing.go:71] worker 59 working on active_alerts task. I0326 23:15:37.794177 1 tasks_processing.go:71] worker 42 working on overlapping_namespace_uids task. W0326 23:15:37.794188 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0326 23:15:37.794203 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 22.283µs to process 0 records I0326 23:15:37.793077 1 tasks_processing.go:71] worker 29 working on storage_classes task. I0326 23:15:37.794296 1 tasks_processing.go:71] worker 48 working on operators_pods_and_events task. I0326 23:15:37.793063 1 tasks_processing.go:71] worker 35 working on pdbs task. I0326 23:15:37.794503 1 tasks_processing.go:71] worker 50 working on image_pruners task. I0326 23:15:37.794517 1 tasks_processing.go:71] worker 27 working on config_maps task. I0326 23:15:37.794568 1 tasks_processing.go:71] worker 38 working on silenced_alerts task. I0326 23:15:37.794588 1 tasks_processing.go:71] worker 36 working on sap_pods task. W0326 23:15:37.794595 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0326 23:15:37.794609 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 22.576µs to process 0 records I0326 23:15:37.794618 1 tasks_processing.go:71] worker 63 working on machines task. I0326 23:15:37.794647 1 tasks_processing.go:71] worker 55 working on image_registries task. I0326 23:15:37.793072 1 tasks_processing.go:71] worker 28 working on openstack_dataplanenodesets task. I0326 23:15:37.793078 1 tasks_processing.go:71] worker 1 working on ingress_certificates task. I0326 23:15:37.794720 1 tasks_processing.go:71] worker 6 working on cluster_apiserver task. I0326 23:15:37.793079 1 tasks_processing.go:71] worker 40 working on machine_healthchecks task. I0326 23:15:37.794770 1 tasks_processing.go:71] worker 43 working on jaegers task. I0326 23:15:37.794871 1 tasks_processing.go:71] worker 56 working on dvo_metrics task. I0326 23:15:37.793079 1 tasks_processing.go:69] worker 60 listening for tasks. I0326 23:15:37.793049 1 tasks_processing.go:69] worker 12 listening for tasks. I0326 23:15:37.793792 1 tasks_processing.go:71] worker 32 working on pod_network_connectivity_checks task. I0326 23:15:37.793034 1 tasks_processing.go:69] worker 47 listening for tasks. I0326 23:15:37.793073 1 tasks_processing.go:71] worker 61 working on feature_gates task. I0326 23:15:37.794038 1 tasks_processing.go:71] worker 58 working on openshift_logging task. I0326 23:15:37.795245 1 tasks_processing.go:71] worker 59 working on ingress task. I0326 23:15:37.795427 1 tasks_processing.go:74] worker 47 stopped. W0326 23:15:37.793091 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0326 23:15:37.795257 1 tasks_processing.go:71] worker 60 working on service_accounts task. I0326 23:15:37.795604 1 tasks_processing.go:74] worker 23 stopped. I0326 23:15:37.795258 1 tasks_processing.go:71] worker 12 working on certificate_signing_requests task. I0326 23:15:37.795624 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 2.53767ms to process 0 records I0326 23:15:37.795265 1 tasks_processing.go:71] worker 38 working on clusterroles task. I0326 23:15:37.797399 1 tasks_processing.go:74] worker 4 stopped. I0326 23:15:37.797411 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 4.440696ms to process 0 records E0326 23:15:37.797418 1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed) I0326 23:15:37.797423 1 tasks_processing.go:74] worker 15 stopped. I0326 23:15:37.797420 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 4.451358ms to process 0 records I0326 23:15:37.797438 1 gather.go:177] gatherer "clusterconfig" function "node_features" took 4.364452ms to process 0 records I0326 23:15:37.797447 1 tasks_processing.go:74] worker 53 stopped. I0326 23:15:37.800810 1 tasks_processing.go:74] worker 24 stopped. I0326 23:15:37.800823 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 7.654873ms to process 0 records I0326 23:15:37.800852 1 tasks_processing.go:74] worker 51 stopped. I0326 23:15:37.800862 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 7.438272ms to process 0 records I0326 23:15:37.800955 1 tasks_processing.go:74] worker 44 stopped. I0326 23:15:37.801037 1 recorder.go:75] Recording config/proxy with fingerprint=cb254ebfbf771573a369f33dfda88aca76bdf27ef312a1f0d8854d420ef91a57 I0326 23:15:37.801039 1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z I0326 23:15:37.801049 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 7.848042ms to process 1 records I0326 23:15:37.801054 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0326 23:15:37.801058 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0326 23:15:37.801061 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0326 23:15:37.801075 1 controller.go:531] The operator is still being initialized I0326 23:15:37.801084 1 controller.go:554] The operator is healthy I0326 23:15:37.801161 1 tasks_processing.go:74] worker 25 stopped. I0326 23:15:37.801359 1 recorder.go:75] Recording config/authentication with fingerprint=fb1dc763f9149d093feb69c9808a1b3252e4e486154807ae759bae99c5932659 I0326 23:15:37.801372 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 7.979878ms to process 1 records I0326 23:15:37.817311 1 tasks_processing.go:74] worker 2 stopped. I0326 23:15:37.817324 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 24.146009ms to process 0 records I0326 23:15:37.817370 1 tasks_processing.go:74] worker 10 stopped. I0326 23:15:37.817384 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 23.523486ms to process 0 records I0326 23:15:37.817393 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 23.574286ms to process 0 records I0326 23:15:37.817401 1 tasks_processing.go:74] worker 9 stopped. I0326 23:15:37.817432 1 tasks_processing.go:74] worker 16 stopped. I0326 23:15:37.817443 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 23.406215ms to process 0 records I0326 23:15:37.817457 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 24.12636ms to process 0 records I0326 23:15:37.817464 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 24.133105ms to process 0 records I0326 23:15:37.817471 1 tasks_processing.go:74] worker 52 stopped. I0326 23:15:37.817475 1 tasks_processing.go:74] worker 8 stopped. I0326 23:15:37.817493 1 tasks_processing.go:74] worker 54 stopped. I0326 23:15:37.817551 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=1cf8338d2d9a8a93ea670b9896046981103feb81243ae69c2cb1e6f20bcb93f7 I0326 23:15:37.817561 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 24.344892ms to process 1 records I0326 23:15:37.817571 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 24.551013ms to process 0 records I0326 23:15:37.817578 1 tasks_processing.go:74] worker 0 stopped. I0326 23:15:37.820935 1 tasks_processing.go:74] worker 17 stopped. I0326 23:15:37.820949 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 27.756155ms to process 0 records I0326 23:15:37.821058 1 tasks_processing.go:74] worker 57 stopped. I0326 23:15:37.821132 1 recorder.go:75] Recording config/image with fingerprint=7415d1ba0db9ae5172959ee8c87eb9fe7ec0ea48d1784d9f3dd31ef8ad3cdaa6 I0326 23:15:37.821142 1 gather.go:177] gatherer "clusterconfig" function "image" took 27.96659ms to process 1 records I0326 23:15:37.826906 1 tasks_processing.go:74] worker 62 stopped. I0326 23:15:37.826922 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 32.855356ms to process 0 records I0326 23:15:37.826940 1 tasks_processing.go:74] worker 18 stopped. I0326 23:15:37.826949 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 33.267905ms to process 0 records I0326 23:15:37.826985 1 tasks_processing.go:74] worker 22 stopped. I0326 23:15:37.826994 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 33.881518ms to process 0 records I0326 23:15:37.827016 1 tasks_processing.go:74] worker 7 stopped. I0326 23:15:37.827033 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 33.722917ms to process 0 records E0326 23:15:37.827043 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0326 23:15:37.827056 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 33.940537ms to process 0 records I0326 23:15:37.827069 1 tasks_processing.go:74] worker 14 stopped. I0326 23:15:37.827103 1 tasks_processing.go:74] worker 30 stopped. I0326 23:15:37.827114 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 34.01302ms to process 0 records I0326 23:15:37.827359 1 tasks_processing.go:74] worker 40 stopped. E0326 23:15:37.827368 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0326 23:15:37.827393 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 32.582921ms to process 0 records I0326 23:15:37.827445 1 tasks_processing.go:74] worker 28 stopped. I0326 23:15:37.827456 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 32.776311ms to process 0 records I0326 23:15:37.829731 1 tasks_processing.go:74] worker 43 stopped. I0326 23:15:37.829743 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 34.950731ms to process 0 records I0326 23:15:37.829764 1 tasks_processing.go:74] worker 32 stopped. E0326 23:15:37.829771 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0326 23:15:37.829777 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 34.873561ms to process 0 records E0326 23:15:37.829783 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0326 23:15:37.829788 1 gather.go:177] gatherer "clusterconfig" function "machines" took 35.147522ms to process 0 records I0326 23:15:37.829795 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 34.650818ms to process 0 records I0326 23:15:37.829800 1 tasks_processing.go:74] worker 58 stopped. I0326 23:15:37.829804 1 tasks_processing.go:74] worker 63 stopped. I0326 23:15:37.829902 1 tasks_processing.go:74] worker 26 stopped. I0326 23:15:37.829972 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 35.82387ms to process 0 records I0326 23:15:37.829981 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 35.341166ms to process 0 records I0326 23:15:37.830007 1 tasks_processing.go:74] worker 36 stopped. I0326 23:15:37.830101 1 tasks_processing.go:74] worker 39 stopped. I0326 23:15:37.830190 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=157613de775ec227d06fc85891d92e1097eda26f533e531216ab1d4611c408d2 I0326 23:15:37.830295 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=e4f140d2672a79ecff52d01070ef097611f7dbc19eceda14d862d91bf718a688 I0326 23:15:37.830318 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=085cf9581a6af1f94224836393058c2a76f11474718893bcfb61a6d9d47175ca I0326 23:15:37.830351 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=ba981307c4815f3a10dc5f4bbce76fa7d1d57b81f9eaf3333e42d05b8474ad51 I0326 23:15:37.830376 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=df38de1ea8e8b636196b30d6da6d0d78089c911119d73b5d97324f08d1892d48 I0326 23:15:37.830840 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=d76b24726acb2acc5d6c987a0fd5a05f29f6b1573657ea12b2089d305d520bcf I0326 23:15:37.830892 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=a1124306239b994841247ff53bfae29a0299b8833eb1fc9886cb5831f93b25aa I0326 23:15:37.830975 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=4297c3d2ede7303b6d3c5627f392b55bac2d89eb6984f027c56348198f5abab3 I0326 23:15:37.831015 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=617ec78475d901ecf92356ce4b82e1d3e7b1468231a0cf953c5074cd2855d1fb I0326 23:15:37.831072 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=e13db6312db4e5b2416985ba34b3984b4814db1e7efa2bc0c4ddf1ebbc5544b4 I0326 23:15:37.831126 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=ed61a322076cec155c2fd8a10dd22c2b1ebad8681adfebf0c6a19aaca7da2dcb I0326 23:15:37.831154 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 36.345829ms to process 11 records I0326 23:15:37.831339 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=b6e43d33dca5044acafba2d0f56f020a75f5c318c9439867f06d45af915dfb91 I0326 23:15:37.831369 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=d615097f2796c36d44f00ec55eee0a8b49e540c318e02168e694df06ed5b85f7 I0326 23:15:37.831394 1 tasks_processing.go:74] worker 35 stopped. I0326 23:15:37.831402 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=faa149d02c0fe896434478ade8edb616cac71a8013cfb0c965fbe3350f87d646 I0326 23:15:37.831427 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 35.621345ms to process 3 records I0326 23:15:37.831535 1 tasks_processing.go:74] worker 6 stopped. I0326 23:15:37.831835 1 recorder.go:75] Recording config/apiserver with fingerprint=7146b1fc1c3a39d78380e65f7f052945203ef40a90841999b09b2fc554f0a489 I0326 23:15:37.831859 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 35.333767ms to process 1 records I0326 23:15:37.832113 1 tasks_processing.go:74] worker 29 stopped. I0326 23:15:37.832138 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=4397a0881ae6d6fdf301b79bfc5f653fd7370b937e7698ddead030f28fa0d353 I0326 23:15:37.832209 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=dc6f8c0e019342a5eb0f7ceac83f4cbc947b4fbcd6f59fa6c38e58b4b1d52032 I0326 23:15:37.832220 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 35.853552ms to process 2 records I0326 23:15:37.832336 1 tasks_processing.go:74] worker 59 stopped. I0326 23:15:37.832617 1 recorder.go:75] Recording config/ingress with fingerprint=fbff1c2e3032c559abada16edd487d4e9bf7517b134f11261058518bb9d85d62 I0326 23:15:37.832655 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 34.809733ms to process 1 records I0326 23:15:37.832675 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 36.114493ms to process 0 records I0326 23:15:37.832726 1 tasks_processing.go:74] worker 11 stopped. I0326 23:15:37.832786 1 tasks_processing.go:74] worker 33 stopped. I0326 23:15:37.832999 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=9a8e78feabd9e193552541cd4cbbfc7ab52d243e8c17881fe89336b9d4edeee3 I0326 23:15:37.833067 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=a38c425c6d4a0fe13448dddd15977011f47c2b68030e01267237c9d419a9bc26 I0326 23:15:37.833120 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=215f910cd5f0ffe6b75f4226b8f18e04c77869870058250da164a42f3547a5e1 I0326 23:15:37.833144 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 36.417418ms to process 3 records I0326 23:15:37.833226 1 tasks_processing.go:74] worker 20 stopped. I0326 23:15:37.833925 1 recorder.go:75] Recording config/oauth with fingerprint=c774ad4774525198cb1f238316280eabc4fcbbbd9c4b4ae966ad1e8d5fce79a1 I0326 23:15:37.833946 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 37.267077ms to process 1 records I0326 23:15:37.834033 1 tasks_processing.go:74] worker 50 stopped. I0326 23:15:37.834206 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=e824c0255c3cf519a8347a91c11a837d4a8b2d9c279fe35fd73cf4074a44343d I0326 23:15:37.834220 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 35.869426ms to process 1 records I0326 23:15:37.834290 1 tasks_processing.go:74] worker 3 stopped. I0326 23:15:37.834790 1 recorder.go:75] Recording config/infrastructure with fingerprint=f6284f5f61125042d6daddff930f32c1b04f0edcce63743ece78678ec174c7a0 I0326 23:15:37.834809 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 37.731922ms to process 1 records I0326 23:15:37.834917 1 recorder.go:75] Recording config/network with fingerprint=97d8ece371ee3f8097e4b82a07dd94ee510f58d7a5cc10ea9e777c4f557f1a95 I0326 23:15:37.834921 1 tasks_processing.go:74] worker 13 stopped. I0326 23:15:37.834935 1 gather.go:177] gatherer "clusterconfig" function "networks" took 37.574436ms to process 1 records I0326 23:15:37.835012 1 recorder.go:75] Recording config/featuregate with fingerprint=157444b3182332960bdda3eced23496f72d4f0807a61a19f5a636d25449cd7ef I0326 23:15:37.835019 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 38.879182ms to process 1 records I0326 23:15:37.835048 1 tasks_processing.go:74] worker 61 stopped. I0326 23:15:37.835113 1 tasks_processing.go:74] worker 41 stopped. I0326 23:15:37.835329 1 recorder.go:75] Recording config/node/ip-10-0-0-167.ec2.internal with fingerprint=f86e182020d0f078ba394cc0ff18e3c1facefa1ff771cf14bad2ab20febf1f02 I0326 23:15:37.835406 1 recorder.go:75] Recording config/node/ip-10-0-1-99.ec2.internal with fingerprint=0d9aa1e24ade13c5884c717fc707405ba8118acf8892018675d057965409fad0 I0326 23:15:37.835497 1 recorder.go:75] Recording config/node/ip-10-0-2-103.ec2.internal with fingerprint=2d16c62b4872bea348e0d786bfedc59ea4a4da9eeb5937ec2ccc227dbc05ea9e I0326 23:15:37.835513 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 39.815565ms to process 3 records I0326 23:15:37.835607 1 tasks_processing.go:74] worker 55 stopped. I0326 23:15:37.835971 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=295855bbfa049dca80819fabb6da9c309e22184f959f1685b6fae14322058e39 I0326 23:15:37.835986 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 39.217367ms to process 1 records E0326 23:15:37.835995 1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io) W0326 23:15:37.836001 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0326 23:15:37.836009 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0326 23:15:37.836017 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 41.403497ms to process 1 records I0326 23:15:37.836028 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0326 23:15:37.836032 1 tasks_processing.go:74] worker 21 stopped. I0326 23:15:37.836062 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 40.858895ms to process 1 records I0326 23:15:37.836072 1 tasks_processing.go:74] worker 42 stopped. I0326 23:15:37.836531 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0326 23:15:37.836596 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0326 23:15:37.836711 1 operator.go:328] started I0326 23:15:37.836731 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0326 23:15:37.837234 1 tasks_processing.go:74] worker 12 stopped. I0326 23:15:37.837249 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 41.605588ms to process 0 records I0326 23:15:37.839761 1 tasks_processing.go:74] worker 34 stopped. I0326 23:15:37.840588 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=b0825d4a6bf24147045989f70c7aafb908168e7213140288ca583495bfd87e4a I0326 23:15:37.840927 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=0e3b35e25053266b1249237623bd2f3492ac1bacb7198c30f9e43b4a68c60a4e I0326 23:15:37.840948 1 gather.go:177] gatherer "clusterconfig" function "crds" took 46.592725ms to process 2 records I0326 23:15:37.842909 1 tasks_processing.go:74] worker 38 stopped. I0326 23:15:37.843065 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=a10a5a1f2a7d9a6cc7b3dfb066ce80a41b5fc0d92dc934a60508ad6ddf2f12d5 I0326 23:15:37.843150 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=dbecaf7f8ae60240fbc1bbb6823b6afb88273f6e0fef118b1b1f923404935875 I0326 23:15:37.843158 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 47.185657ms to process 2 records I0326 23:15:37.843611 1 controller.go:254] Source scaController *sca.Controller is not ready I0326 23:15:37.843623 1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready I0326 23:15:37.843628 1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0326 23:15:37.843631 1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready I0326 23:15:37.843634 1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready I0326 23:15:37.843648 1 controller.go:531] The operator is still being initialized I0326 23:15:37.843655 1 controller.go:554] The operator is healthy I0326 23:15:37.843859 1 tasks_processing.go:74] worker 37 stopped. I0326 23:15:37.845021 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-8777bd9bb-wj8cq with fingerprint=b57cb8576a218c80e0089af5ffbf6b07694363e319af499cb769cae9da015624 I0326 23:15:37.845290 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-5zdwv with fingerprint=a946f10a611af54ecdfa5844e39403c980357217a190d3c7dcc512aeceeed1de I0326 23:15:37.845337 1 recorder.go:75] Recording config/running_containers with fingerprint=06301850a142b048e9ad2425722ec5a0dbd1be57a219abdf22e1201b16412bb0 I0326 23:15:37.845346 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 50.735098ms to process 3 records I0326 23:15:37.851472 1 prometheus_rules.go:88] Prometheus rules successfully created I0326 23:15:37.854381 1 tasks_processing.go:74] worker 49 stopped. I0326 23:15:37.854394 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 61.166144ms to process 0 records E0326 23:15:37.862834 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27babdc3b0-0295-47a5-a13e-e4b06fb9ebd2%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:45363->172.30.0.10:53: read: connection refused I0326 23:15:37.862848 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27babdc3b0-0295-47a5-a13e-e4b06fb9ebd2%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:45363->172.30.0.10:53: read: connection refused I0326 23:15:37.863867 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0326 23:15:37.863874 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0326 23:15:37.863893 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0326 23:15:37.867082 1 base_controller.go:82] Caches are synced for ConfigController I0326 23:15:37.867094 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0326 23:15:37.867791 1 tasks_processing.go:74] worker 1 stopped. E0326 23:15:37.867804 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0326 23:15:37.867811 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2p9mp5cv8t0e1bpnh305hr8ld9shop2c-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2p9mp5cv8t0e1bpnh305hr8ld9shop2c-primary-cert-bundle-secret" not found I0326 23:15:37.867858 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=8620065dbfcf8cc41e546c504930f37ad4b49444cf1d8194abce7d1729798d58 I0326 23:15:37.867870 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 73.065533ms to process 1 records I0326 23:15:37.868298 1 tasks_processing.go:74] worker 27 stopped. E0326 23:15:37.868312 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0326 23:15:37.868317 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0326 23:15:37.868321 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0326 23:15:37.868329 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=7c953bef0cdbddb25a47a3c40772dfa7cbf73ae2ac9bb3dc97984dbf1bd43499 I0326 23:15:37.868368 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0326 23:15:37.868376 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0326 23:15:37.868380 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=4bec6f0bfa45f0ccf5709371166150dcb9cf2703d04f5f6de4476d0277c477a6 I0326 23:15:37.868384 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0326 23:15:37.868420 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0326 23:15:37.868427 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0326 23:15:37.868433 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 73.764776ms to process 7 records I0326 23:15:37.870868 1 configmapobserver.go:84] configmaps "insights-config" not found I0326 23:15:37.908672 1 tasks_processing.go:74] worker 5 stopped. I0326 23:15:37.908679 1 gather_logs.go:145] no pods in namespace were found I0326 23:15:37.908918 1 recorder.go:75] Recording config/version with fingerprint=f82b14029ad1bea1fb21d6c4d1cfd283997ce3080c28e95813bfe2c3b0768e15 I0326 23:15:37.908936 1 recorder.go:75] Recording config/id with fingerprint=7c000abe07b2706ce17fd19ae730b59cfd2578d6fbef28707ac9cff8e3aa6153 I0326 23:15:37.908942 1 gather.go:177] gatherer "clusterconfig" function "version" took 115.306574ms to process 2 records I0326 23:15:37.908953 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 114.886633ms to process 0 records I0326 23:15:37.908960 1 tasks_processing.go:74] worker 19 stopped. I0326 23:15:37.923555 1 tasks_processing.go:74] worker 45 stopped. I0326 23:15:37.923570 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 130.467043ms to process 0 records I0326 23:15:37.932215 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0326 23:15:37.935469 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:56038->172.30.0.10:53: read: connection refused I0326 23:15:37.935482 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.9:56038->172.30.0.10:53: read: connection refused I0326 23:15:37.937561 1 base_controller.go:82] Caches are synced for LoggingSyncer I0326 23:15:37.937573 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0326 23:15:38.253017 1 gather_cluster_operator_pods_and_events.go:121] Found 18 pods with 21 containers I0326 23:15:38.253030 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1198372 bytes I0326 23:15:38.253208 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-45sch pod in namespace openshift-dns (previous: false). I0326 23:15:38.470775 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-45sch pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-45sch\" is waiting to start: ContainerCreating" I0326 23:15:38.470794 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-45sch\" is waiting to start: ContainerCreating" I0326 23:15:38.470802 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-45sch pod in namespace openshift-dns (previous: false). I0326 23:15:38.646406 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-45sch pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-45sch\" is waiting to start: ContainerCreating" I0326 23:15:38.646421 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-45sch\" is waiting to start: ContainerCreating" I0326 23:15:38.646432 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-q2mms pod in namespace openshift-dns (previous: false). W0326 23:15:38.834495 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0326 23:15:38.889310 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-q2mms pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-q2mms\" is waiting to start: ContainerCreating" I0326 23:15:38.889327 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-q2mms\" is waiting to start: ContainerCreating" I0326 23:15:38.889336 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-q2mms pod in namespace openshift-dns (previous: false). I0326 23:15:39.048104 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-q2mms pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-q2mms\" is waiting to start: ContainerCreating" I0326 23:15:39.048118 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-q2mms\" is waiting to start: ContainerCreating" I0326 23:15:39.048136 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-z86q2 pod in namespace openshift-dns (previous: false). I0326 23:15:39.257065 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0326 23:15:39.268140 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-z86q2 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-z86q2\" is waiting to start: ContainerCreating" I0326 23:15:39.268154 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-z86q2\" is waiting to start: ContainerCreating" I0326 23:15:39.268162 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-z86q2 pod in namespace openshift-dns (previous: false). I0326 23:15:39.446672 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-z86q2 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-z86q2\" is waiting to start: ContainerCreating" I0326 23:15:39.446691 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-z86q2\" is waiting to start: ContainerCreating" I0326 23:15:39.446702 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-54sx7 pod in namespace openshift-dns (previous: false). I0326 23:15:39.647303 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0326 23:15:39.647322 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-ff4r4 pod in namespace openshift-dns (previous: false). W0326 23:15:39.834968 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0326 23:15:39.857542 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0326 23:15:39.857558 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-fps5g pod in namespace openshift-dns (previous: false). I0326 23:15:39.858166 1 tasks_processing.go:74] worker 31 stopped. I0326 23:15:39.858224 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=3372cad4a0956bedefe9f9912cae18fdf6b7bb657feb12c953fae610872ee075 I0326 23:15:39.858297 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=2583c1e6c8dfc14cb27dd1bbb09423f3e855961d52c46d65309e31cf47e20db2 I0326 23:15:39.858337 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0326 23:15:39.858380 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=a71b9428d4d102e6ec72a90baa2e4a9ffb575ed17dc33b15ce4469b2699f4bd6 I0326 23:15:39.858427 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0326 23:15:39.858463 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=f6268ba2a5ec90204679adb6c3b822c1c18c7986ab7dafee87efbb159b5e5c1e I0326 23:15:39.858505 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=84103887ef93f921238c94aee8e64da44fb386d4525718d80bbaee5990ce77a1 I0326 23:15:39.858540 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=e735d097decb34b8c2f0cf776439b9b7a75440f128e21a0a90f1f43a1eeba4b5 I0326 23:15:39.858563 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=efa0b615a4d780df6ffb72f24d4f11cb6df2ab4ebaa8047a5092374426e0dddf I0326 23:15:39.858593 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=6e51b080958ad54b784b951f59396f66a6f3d5d154b8dbe1ce570d9ec9eb1b9a I0326 23:15:39.858607 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0326 23:15:39.858636 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=555915db62f4bae0f2eda9b293c5396cabfb2cceec565127afc6fa59a94dd27b I0326 23:15:39.858654 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0326 23:15:39.858679 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=479dc296faf7773c70c61ce002b1b070083ede24facc1ea2f053e365c8de8ccd I0326 23:15:39.858693 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0326 23:15:39.858717 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=9b7334d2028a9434a953b770706b114489a2db92fb5e457f65fc4638faea8c2e I0326 23:15:39.858731 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0326 23:15:39.858757 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=c3051d1f47b00101d924ca7b2bee42b2e081610270b73c33513722c3979d4678 I0326 23:15:39.858909 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=79ad67a15e2c84841bc8bd91ce245424a2c2e8feacb7462783bb71783dc4ebb6 I0326 23:15:39.858926 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0326 23:15:39.858939 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0326 23:15:39.858970 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0326 23:15:39.859005 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=034e69b432a03da1cc7c1f1d94a4cebe1241540ea3de4ea3adc2a1febb904204 I0326 23:15:39.859043 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=cabd0aba582eccfbb0d8875cfdcfdb63b940c68419bff73cee88882f9f05f20d I0326 23:15:39.859058 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0326 23:15:39.859095 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=b8e86c7b7495a6abb3366d613f1aa4473edfe9fae5261aee7d7289a461d2b23b I0326 23:15:39.859110 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0326 23:15:39.859133 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=ad2a7c8532914d8b6cc662e0f9df482ff9757df09bf7bf520b901b5c61e289e5 I0326 23:15:39.859157 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=103606b11cd7144b9703cc4ca23d520654542e7fc55112ce0cdcc688be2d9e69 I0326 23:15:39.859180 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=f1d49a4df8b3c52103562e59d77444e9481da83636e95fe3edc4a78c0ea027f7 I0326 23:15:39.859208 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=de1b29dfb6e6e6e429ee38cbbe0b7356ad0a44f94cd723e3440675fb60e334f0 I0326 23:15:39.859241 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=965f995db09e9b3f2be06c5f27fcb4a181b97c646979c61c918f892f281b6b71 I0326 23:15:39.859254 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0326 23:15:39.859308 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=c56bede25dddfa0b47ac30a69b0924bef63754d3260728ebbe966d67264b242a I0326 23:15:39.859335 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0326 23:15:39.859348 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0326 23:15:39.859359 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.064938254s to process 36 records I0326 23:15:40.047281 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0326 23:15:40.047300 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-56599b6895-b246v pod in namespace openshift-image-registry (previous: false). I0326 23:15:40.246044 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-56599b6895-b246v pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-56599b6895-b246v\" is waiting to start: ContainerCreating" I0326 23:15:40.246062 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-56599b6895-b246v\" is waiting to start: ContainerCreating" I0326 23:15:40.246078 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-56599b6895-wp4tk pod in namespace openshift-image-registry (previous: false). I0326 23:15:40.450888 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-56599b6895-wp4tk pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-56599b6895-wp4tk\" is waiting to start: ContainerCreating" I0326 23:15:40.450908 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-56599b6895-wp4tk\" is waiting to start: ContainerCreating" I0326 23:15:40.450921 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-574cbc9d5b-54cdm pod in namespace openshift-image-registry (previous: false). I0326 23:15:40.648951 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-574cbc9d5b-54cdm pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-574cbc9d5b-54cdm\" is waiting to start: ContainerCreating" I0326 23:15:40.648969 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-574cbc9d5b-54cdm\" is waiting to start: ContainerCreating" I0326 23:15:40.648980 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-7vl56 pod in namespace openshift-image-registry (previous: false). W0326 23:15:40.835488 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0326 23:15:40.851639 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0326 23:15:40.851654 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-qltrr pod in namespace openshift-image-registry (previous: false). I0326 23:15:41.047817 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0326 23:15:41.047837 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-x6qrj pod in namespace openshift-image-registry (previous: false). I0326 23:15:41.247950 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0326 23:15:41.247969 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6565674565-gspwq pod in namespace openshift-ingress (previous: false). I0326 23:15:41.447549 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6565674565-gspwq pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6565674565-gspwq\" is waiting to start: ContainerCreating" I0326 23:15:41.447568 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6565674565-gspwq\" is waiting to start: ContainerCreating" I0326 23:15:41.447582 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7c965c98cf-gqmmz pod in namespace openshift-ingress (previous: false). I0326 23:15:41.648862 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7c965c98cf-gqmmz pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7c965c98cf-gqmmz\" is waiting to start: ContainerCreating" I0326 23:15:41.648880 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7c965c98cf-gqmmz\" is waiting to start: ContainerCreating" I0326 23:15:41.648893 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7c965c98cf-ts7zq pod in namespace openshift-ingress (previous: false). W0326 23:15:41.835414 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0326 23:15:41.847789 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7c965c98cf-ts7zq pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7c965c98cf-ts7zq\" is waiting to start: ContainerCreating" I0326 23:15:41.847809 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7c965c98cf-ts7zq\" is waiting to start: ContainerCreating" I0326 23:15:41.847820 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-8dhvq pod in namespace openshift-ingress-canary (previous: false). I0326 23:15:42.048118 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-h4v5c pod in namespace openshift-ingress-canary (previous: false). I0326 23:15:42.247718 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-mfvl8 pod in namespace openshift-ingress-canary (previous: false). I0326 23:15:42.448955 1 tasks_processing.go:74] worker 48 stopped. I0326 23:15:42.449058 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=fff6cdc7194c8dd0d865711d067d19d4b0944aca152a1cb11cf432e6edb6849a I0326 23:15:42.449115 1 recorder.go:75] Recording events/openshift-dns with fingerprint=0d8eac3595ef9ce43a1486b1165b234b68e3aedfa53888b5f35244783394fc92 I0326 23:15:42.449209 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=3a4414bb54d3f521667da33a4b638a34b818fb4d8e9b9bcfeaae8445df6a4686 I0326 23:15:42.449237 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=b1972ae45f180b448faafd1d3d86e568db1878bbf127912f6c11ae7857203293 I0326 23:15:42.449297 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=8877188fc7ac0c7a5a6608b74b059e4ffb3b317b8f9007dec3157b347859287e I0326 23:15:42.449335 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=ae1b0699b49d575cd7a3e09e282bb973ae8455c8386ff34492ed1efb038cbf45 I0326 23:15:42.449342 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-8dhvq/serve-healthcheck-canary_current.log with fingerprint=db6d02fffb6a45589b25b22336c6d9412f286634b1aa84bbff0eedcd3a9900bc I0326 23:15:42.449347 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-h4v5c/serve-healthcheck-canary_current.log with fingerprint=3201585218f8b8a79f512809355f020b6211f3cd26ed0bf40d803364366fc487 I0326 23:15:42.449351 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-mfvl8/serve-healthcheck-canary_current.log with fingerprint=26751000f95ef12a5e5c2082d918f51163e50907222121613d75fa89259c0e6b I0326 23:15:42.449358 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.654640116s to process 9 records W0326 23:15:42.835367 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0326 23:15:42.835391 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0326 23:15:42.835406 1 tasks_processing.go:74] worker 56 stopped. E0326 23:15:42.835417 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0326 23:15:42.835427 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0326 23:15:42.835442 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0326 23:15:42.835454 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.04051808s to process 1 records I0326 23:15:50.239109 1 tasks_processing.go:74] worker 46 stopped. I0326 23:15:50.239150 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0326 23:15:50.239164 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.445075562s to process 1 records I0326 23:15:50.751187 1 configmapobserver.go:84] configmaps "insights-config" not found I0326 23:15:51.002072 1 tasks_processing.go:74] worker 60 stopped. I0326 23:15:51.002382 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=7c1a3e3c431de69ce4ee6fb33aad2158a3d38446a91ec5899ca71ceaf725097e I0326 23:15:51.002404 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.206570626s to process 1 records E0326 23:15:51.002465 1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.209s with: function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machines\" failed with an error, function \"machine_configs\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0326 23:15:51.003573 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machines" failed with an error, function "machine_configs" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0326 23:15:51.003589 1 periodic.go:216] Running workloads gatherer I0326 23:15:51.003606 1 tasks_processing.go:45] number of workers: 2 I0326 23:15:51.003612 1 tasks_processing.go:69] worker 1 listening for tasks. I0326 23:15:51.003616 1 tasks_processing.go:71] worker 1 working on workload_info task. I0326 23:15:51.003622 1 tasks_processing.go:69] worker 0 listening for tasks. I0326 23:15:51.003646 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0326 23:15:51.025018 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0326 23:15:51.026452 1 tasks_processing.go:74] worker 0 stopped. I0326 23:15:51.026464 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 22.78481ms to process 0 records I0326 23:15:51.033040 1 gather_workloads_info.go:387] No image sha256:0fd69333c121efd68919e731cd3ea01e976e00415bda6e987bf1e949ec224eda (9ms) I0326 23:15:51.040976 1 gather_workloads_info.go:387] No image sha256:273930332d7b481bbca4521cf740507498a2a323a4010cd66e896806b299f85e (8ms) I0326 23:15:51.048201 1 gather_workloads_info.go:387] No image sha256:72ae38682f25786aec18625cfed5d7831017d2f0f1304c3320c00ce0b7fb09c1 (7ms) I0326 23:15:51.055925 1 gather_workloads_info.go:387] No image sha256:a6b085cdad58db8529d99c22ec1ec79fe09ab00ab1c92ee38b29e7875dc0533c (8ms) I0326 23:15:51.063333 1 gather_workloads_info.go:387] No image sha256:31e175b644c6627779ca6b9f48d9426760dd449788ad671dd0277cce8fb822e8 (7ms) I0326 23:15:51.070613 1 gather_workloads_info.go:387] No image sha256:6ea0f19030ffc52a290e6eeb812ef2426c48a2f1083b1e415e0667998cbb7bf2 (7ms) I0326 23:15:51.078427 1 gather_workloads_info.go:387] No image sha256:2e3661304917ce7f3be49974faacc60c24cbe2833d7b50c8c205c2f3f2467950 (8ms) I0326 23:15:51.087053 1 gather_workloads_info.go:387] No image sha256:66640a3f7bd65090345552611701a73e74bc44897b9d29e43b11f73b96598ac0 (9ms) I0326 23:15:51.094468 1 gather_workloads_info.go:387] No image sha256:12aec97ddeb527881acf5949bea2b227baa1dbfccf30679e8d57ff0b82e42707 (7ms) I0326 23:15:51.101770 1 gather_workloads_info.go:387] No image sha256:1e6d59867cbd750672bc9ccfb9c69c3d198174cd43dd6d71a0da358b4b766473 (7ms) I0326 23:15:51.133537 1 gather_workloads_info.go:387] No image sha256:2ade6dfb5406ef3a4264dea95d455298f54e8ef4d458d9ee3e94c33d48676a9b (32ms) I0326 23:15:51.235981 1 gather_workloads_info.go:387] No image sha256:8f72efd51f564d8d3e4ecee6555d7bd5394d0776054d2112febea6b7c9e71225 (102ms) I0326 23:15:51.333234 1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (97ms) I0326 23:15:51.434167 1 gather_workloads_info.go:387] No image sha256:6263faa8ef7759ec21f26df08314c5d64e52d967b49961cb9404c54532c03365 (101ms) I0326 23:15:51.533400 1 gather_workloads_info.go:387] No image sha256:20a3e59566dd25058fff22e18569067fe6a12d8ac5ac62fa6508e0cee44d7897 (99ms) I0326 23:15:51.633078 1 gather_workloads_info.go:387] No image sha256:cfcb0bd30c537e677ac6856c36c0c923b415f24d944c7aa1386489f511f6491d (100ms) I0326 23:15:51.738515 1 gather_workloads_info.go:387] No image sha256:83344c7d1374cbc2607b4fcb837f094c035b32daf24f8b3ccba2c8df409a6d3c (105ms) I0326 23:15:51.835457 1 gather_workloads_info.go:387] No image sha256:3739c0c55f7f7ccc84309ae11eb23d3d9e5ad50a68e849a8a8dfda7f2f2bc137 (97ms) I0326 23:15:51.933244 1 gather_workloads_info.go:387] No image sha256:1b58ba57c10e2ba03a0d8dba1f3f8faa866e1dca6170725b6909c2cc2b96efa6 (98ms) I0326 23:15:52.033012 1 gather_workloads_info.go:387] No image sha256:87de118dc29ca1b47d4768ec5ed4441bc1dada3fb2a78d53b1833249baa86dd4 (100ms) I0326 23:15:52.133733 1 gather_workloads_info.go:387] No image sha256:51e1b5dc9fccbee5b7df78d0e8bbe379f75529232c102c9ed5b1ae134c4b9b4c (101ms) I0326 23:15:52.233817 1 gather_workloads_info.go:387] No image sha256:aa65d1c7fb8805b767fc4878faea280472f372010efadc66e5641a2e634a80c0 (100ms) I0326 23:15:52.332747 1 gather_workloads_info.go:387] No image sha256:5b51a4d9d3b7ecad110fb6eb00f71e9e9b854067d79b220df397a58d9fa50eb0 (99ms) I0326 23:15:52.433323 1 gather_workloads_info.go:387] No image sha256:2f26472542da4f89a2ac34a93fd438d385419dda16811bdfa7e679e0115abe56 (101ms) I0326 23:15:52.533082 1 gather_workloads_info.go:387] No image sha256:4e62a2dcb8214491388d4c17f89fe699eb53cdb8039dcba6ab052edc28da3fbd (100ms) I0326 23:15:52.639771 1 gather_workloads_info.go:387] No image sha256:12448aad6e9050c2b2445035be818a490cef502d39b73dbd51cc0f5ea7d67d40 (107ms) I0326 23:15:52.733562 1 gather_workloads_info.go:387] No image sha256:4c39dc633bf3560c2397281b74faf9d00500b0738dec6c1aad2cc8ce326be6fc (94ms) I0326 23:15:52.833708 1 gather_workloads_info.go:387] No image sha256:a20df9dd8819885aa431022c6f5e80b77b4431f268b719d71bec26f10521da0e (100ms) I0326 23:15:52.933715 1 gather_workloads_info.go:387] No image sha256:3da5a449ace61cb7d002d8e7911c14905c74cf58a296527458d1af0f1abf0a72 (100ms) I0326 23:15:53.033215 1 gather_workloads_info.go:387] No image sha256:185a95b130b6a27ad88fcf39803215b7bd998472b1e37225858435f4eb7c3c04 (99ms) I0326 23:15:53.088656 1 configmapobserver.go:84] configmaps "insights-config" not found I0326 23:15:53.133017 1 gather_workloads_info.go:387] No image sha256:1c5b548d67e5c56e3297b2f8aa34428e544b5b5ac88bd8d0f465760375c501e6 (100ms) I0326 23:15:53.233582 1 gather_workloads_info.go:387] No image sha256:ad0e262327997e8cdc4074a2373e67274d9b8a943aa274a2cd8ce98ffe1db4a0 (101ms) I0326 23:15:53.233611 1 tasks_processing.go:74] worker 1 stopped. E0326 23:15:53.233621 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0326 23:15:53.233854 1 recorder.go:75] Recording config/workload_info with fingerprint=140222a983c8c9c75f3eb4c5adce369a14b3c975cae191e5a2f769bb1984207c I0326 23:15:53.233868 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.229988806s to process 1 records E0326 23:15:53.233892 1 periodic.go:254] "Unhandled Error" err="workloads failed after 2.23s with: function \"workload_info\" failed with an error" I0326 23:15:53.234994 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0326 23:15:53.235006 1 periodic.go:216] Running conditional gatherer I0326 23:15:53.240147 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.6/gathering_rules I0326 23:15:53.246873 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.6/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.9:45481->172.30.0.10:53: read: connection refused E0326 23:15:53.247178 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0326 23:15:53.247254 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0326 23:15:53.252214 1 conditional_gatherer.go:392] cluster version is '4.21.6' E0326 23:15:53.252226 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252231 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252234 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252237 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252240 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252244 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252248 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252251 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0326 23:15:53.252253 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0326 23:15:53.252284 1 tasks_processing.go:45] number of workers: 3 I0326 23:15:53.252301 1 tasks_processing.go:69] worker 2 listening for tasks. I0326 23:15:53.252307 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0326 23:15:53.252309 1 tasks_processing.go:69] worker 0 listening for tasks. I0326 23:15:53.252317 1 tasks_processing.go:69] worker 1 listening for tasks. I0326 23:15:53.252334 1 tasks_processing.go:74] worker 1 stopped. I0326 23:15:53.252343 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0326 23:15:53.252387 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0326 23:15:53.252400 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.173µs to process 1 records I0326 23:15:53.252345 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0326 23:15:53.252435 1 tasks_processing.go:74] worker 0 stopped. I0326 23:15:53.252498 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0326 23:15:53.252513 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 4.088µs to process 1 records I0326 23:15:53.252618 1 tasks_processing.go:74] worker 2 stopped. I0326 23:15:53.252631 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 262.69µs to process 0 records I0326 23:15:53.252649 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.6/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.9:45481->172.30.0.10:53: read: connection refused I0326 23:15:53.252665 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0326 23:15:53.272915 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=224345465effa53174103e721ea2dfb15978fa629e1eadbf10265c4073f0f4b6 I0326 23:15:53.273026 1 diskrecorder.go:70] Writing 105 records to /var/lib/insights-operator/insights-2026-03-26-231553.tar.gz I0326 23:15:53.281250 1 diskrecorder.go:51] Wrote 105 records to disk in 8ms I0326 23:15:53.281295 1 periodic.go:285] Gathering cluster info every 2h0m0s I0326 23:15:53.281310 1 periodic.go:286] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0326 23:15:53.289546 1 configmapobserver.go:84] configmaps "insights-config" not found I0326 23:16:00.972986 1 configmapobserver.go:84] configmaps "insights-config" not found I0326 23:17:02.368184 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="d4844b9f1aba11e5221b19d7f1c9c20b968afd9f4ae68f5c106d30593152a75c") W0326 23:17:02.368221 1 builder.go:160] Restart triggered because of file /var/run/configmaps/service-ca-bundle/service-ca.crt was created I0326 23:17:02.368302 1 periodic.go:177] Shutting down