W0216 18:13:13.879756 1 cmd.go:257] Using insecure, self-signed certificates I0216 18:13:14.906196 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0216 18:13:14.906551 1 observer_polling.go:159] Starting file observer I0216 18:13:15.456170 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0216 18:13:15.456461 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0216 18:13:15.457369 1 secure_serving.go:57] Forcing use of http/1.1 only W0216 18:13:15.457398 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0216 18:13:15.457404 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0216 18:13:15.457409 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0216 18:13:15.457413 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0216 18:13:15.457417 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0216 18:13:15.457421 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0216 18:13:15.457445 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0216 18:13:15.462516 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0216 18:13:15.462566 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"abe00503-996d-4eda-9527-fc52644d2599", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0216 18:13:15.463769 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0216 18:13:15.463788 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0216 18:13:15.463802 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0216 18:13:15.463824 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0216 18:13:15.464275 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0216 18:13:15.464301 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0216 18:13:15.465056 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-3028609660/tls.crt::/tmp/serving-cert-3028609660/tls.key" I0216 18:13:15.465560 1 secure_serving.go:213] Serving securely on [::]:8443 I0216 18:13:15.465574 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0216 18:13:15.469652 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0216 18:13:15.469699 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0216 18:13:15.469784 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0216 18:13:15.475495 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0216 18:13:15.475526 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0216 18:13:15.480568 1 secretconfigobserver.go:119] support secret does not exist I0216 18:13:15.485912 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0216 18:13:15.491071 1 secretconfigobserver.go:119] support secret does not exist I0216 18:13:15.493823 1 recorder.go:161] Pruning old reports every 8h38m14s, max age is 288h0m0s I0216 18:13:15.499719 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0216 18:13:15.499723 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0216 18:13:15.499743 1 insightsreport.go:296] Starting report retriever I0216 18:13:15.499741 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0216 18:13:15.499749 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0216 18:13:15.499719 1 periodic.go:209] Running clusterconfig gatherer I0216 18:13:15.499829 1 tasks_processing.go:45] number of workers: 32 I0216 18:13:15.499876 1 tasks_processing.go:69] worker 7 listening for tasks. I0216 18:13:15.499887 1 tasks_processing.go:69] worker 8 listening for tasks. I0216 18:13:15.499891 1 tasks_processing.go:71] worker 7 working on schedulers task. I0216 18:13:15.499895 1 tasks_processing.go:69] worker 0 listening for tasks. I0216 18:13:15.499899 1 tasks_processing.go:69] worker 20 listening for tasks. I0216 18:13:15.499905 1 tasks_processing.go:69] worker 21 listening for tasks. I0216 18:13:15.499905 1 tasks_processing.go:69] worker 19 listening for tasks. I0216 18:13:15.499911 1 tasks_processing.go:69] worker 1 listening for tasks. I0216 18:13:15.499919 1 tasks_processing.go:69] worker 2 listening for tasks. I0216 18:13:15.499921 1 tasks_processing.go:69] worker 24 listening for tasks. I0216 18:13:15.499913 1 tasks_processing.go:69] worker 22 listening for tasks. I0216 18:13:15.499917 1 tasks_processing.go:69] worker 23 listening for tasks. I0216 18:13:15.499929 1 tasks_processing.go:69] worker 27 listening for tasks. I0216 18:13:15.499931 1 tasks_processing.go:69] worker 9 listening for tasks. I0216 18:13:15.499935 1 tasks_processing.go:69] worker 28 listening for tasks. I0216 18:13:15.499937 1 tasks_processing.go:69] worker 25 listening for tasks. I0216 18:13:15.499942 1 tasks_processing.go:69] worker 10 listening for tasks. I0216 18:13:15.499941 1 tasks_processing.go:69] worker 26 listening for tasks. I0216 18:13:15.499947 1 tasks_processing.go:69] worker 29 listening for tasks. I0216 18:13:15.499952 1 tasks_processing.go:69] worker 11 listening for tasks. I0216 18:13:15.499953 1 tasks_processing.go:69] worker 30 listening for tasks. I0216 18:13:15.499959 1 tasks_processing.go:69] worker 15 listening for tasks. I0216 18:13:15.499962 1 tasks_processing.go:69] worker 16 listening for tasks. I0216 18:13:15.499963 1 tasks_processing.go:69] worker 17 listening for tasks. I0216 18:13:15.499968 1 tasks_processing.go:69] worker 12 listening for tasks. I0216 18:13:15.499970 1 tasks_processing.go:69] worker 18 listening for tasks. I0216 18:13:15.499973 1 tasks_processing.go:69] worker 6 listening for tasks. I0216 18:13:15.499977 1 tasks_processing.go:69] worker 13 listening for tasks. I0216 18:13:15.499981 1 tasks_processing.go:69] worker 4 listening for tasks. I0216 18:13:15.499981 1 tasks_processing.go:69] worker 5 listening for tasks. I0216 18:13:15.499984 1 tasks_processing.go:69] worker 14 listening for tasks. I0216 18:13:15.499878 1 tasks_processing.go:69] worker 31 listening for tasks. I0216 18:13:15.499992 1 tasks_processing.go:71] worker 0 working on storage_cluster task. I0216 18:13:15.499997 1 tasks_processing.go:71] worker 8 working on proxies task. I0216 18:13:15.500001 1 tasks_processing.go:71] worker 31 working on container_runtime_configs task. I0216 18:13:15.500154 1 tasks_processing.go:71] worker 26 working on ingress task. I0216 18:13:15.500201 1 tasks_processing.go:71] worker 12 working on silenced_alerts task. I0216 18:13:15.500211 1 tasks_processing.go:71] worker 6 working on cost_management_metrics_configs task. W0216 18:13:15.500232 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0216 18:13:15.500245 1 tasks_processing.go:71] worker 12 working on openshift_logging task. I0216 18:13:15.500290 1 tasks_processing.go:71] worker 20 working on storage_classes task. I0216 18:13:15.500208 1 tasks_processing.go:71] worker 18 working on support_secret task. I0216 18:13:15.499885 1 tasks_processing.go:69] worker 3 listening for tasks. I0216 18:13:15.500681 1 tasks_processing.go:71] worker 3 working on jaegers task. I0216 18:13:15.500333 1 tasks_processing.go:71] worker 4 working on openshift_machine_api_events task. I0216 18:13:15.500338 1 tasks_processing.go:71] worker 13 working on openstack_dataplanedeployments task. I0216 18:13:15.500349 1 tasks_processing.go:71] worker 22 working on dvo_metrics task. I0216 18:13:15.500343 1 tasks_processing.go:71] worker 14 working on qemu_kubevirt_launcher_logs task. I0216 18:13:15.500353 1 tasks_processing.go:71] worker 21 working on image_pruners task. I0216 18:13:15.500354 1 tasks_processing.go:71] worker 15 working on aggregated_monitoring_cr_names task. I0216 18:13:15.500348 1 tasks_processing.go:71] worker 5 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0216 18:13:15.500358 1 tasks_processing.go:71] worker 19 working on overlapping_namespace_uids task. I0216 18:13:15.500359 1 tasks_processing.go:71] worker 29 working on machines task. I0216 18:13:15.500361 1 tasks_processing.go:71] worker 2 working on tsdb_status task. W0216 18:13:15.503246 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0216 18:13:15.500364 1 tasks_processing.go:71] worker 11 working on node_logs task. I0216 18:13:15.500363 1 tasks_processing.go:71] worker 24 working on machine_configs task. I0216 18:13:15.500373 1 tasks_processing.go:71] worker 17 working on machine_config_pools task. I0216 18:13:15.500368 1 tasks_processing.go:71] worker 1 working on machine_healthchecks task. I0216 18:13:15.500368 1 tasks_processing.go:71] worker 30 working on certificate_signing_requests task. I0216 18:13:15.500379 1 tasks_processing.go:71] worker 9 working on sap_datahubs task. I0216 18:13:15.500379 1 tasks_processing.go:71] worker 25 working on pod_network_connectivity_checks task. I0216 18:13:15.500382 1 tasks_processing.go:71] worker 23 working on crds task. I0216 18:13:15.500377 1 tasks_processing.go:71] worker 16 working on image_registries task. I0216 18:13:15.500382 1 tasks_processing.go:71] worker 28 working on metrics task. I0216 18:13:15.500386 1 tasks_processing.go:71] worker 27 working on openstack_dataplanenodesets task. I0216 18:13:15.500388 1 tasks_processing.go:71] worker 10 working on sap_config task. I0216 18:13:15.500398 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 29.541µs to process 0 records I0216 18:13:15.503350 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 104.134µs to process 0 records I0216 18:13:15.503372 1 tasks_processing.go:71] worker 2 working on operators task. W0216 18:13:15.503379 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0216 18:13:15.503403 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 31.268µs to process 0 records I0216 18:13:15.503572 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 3.55832ms to process 0 records I0216 18:13:15.503611 1 tasks_processing.go:71] worker 31 working on machine_sets task. I0216 18:13:15.504109 1 tasks_processing.go:71] worker 28 working on mutating_webhook_configurations task. I0216 18:13:15.504454 1 tasks_processing.go:71] worker 12 working on nodenetworkconfigurationpolicies task. I0216 18:13:15.504497 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 4.198125ms to process 0 records I0216 18:13:15.504625 1 tasks_processing.go:71] worker 6 working on monitoring_persistent_volumes task. I0216 18:13:15.504740 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 4.397972ms to process 0 records I0216 18:13:15.505857 1 tasks_processing.go:71] worker 0 working on service_accounts task. I0216 18:13:15.505897 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 5.855505ms to process 0 records I0216 18:13:15.505914 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 5.194374ms to process 0 records I0216 18:13:15.505939 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 5.084305ms to process 0 records I0216 18:13:15.505956 1 tasks_processing.go:71] worker 13 working on authentication task. I0216 18:13:15.505961 1 tasks_processing.go:71] worker 3 working on infrastructures task. I0216 18:13:15.506087 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0216 18:13:15.506111 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0216 18:13:15.506119 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0216 18:13:15.506124 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0216 18:13:15.506144 1 controller.go:489] The operator is still being initialized I0216 18:13:15.506195 1 controller.go:512] The operator is healthy I0216 18:13:15.508713 1 tasks_processing.go:71] worker 18 working on clusterroles task. E0216 18:13:15.508725 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0216 18:13:15.508746 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 8.14791ms to process 0 records I0216 18:13:15.508772 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 8.005577ms to process 0 records I0216 18:13:15.508854 1 tasks_processing.go:71] worker 4 working on version task. I0216 18:13:15.509006 1 tasks_processing.go:71] worker 26 working on machine_autoscalers task. I0216 18:13:15.509353 1 recorder.go:75] Recording config/ingress with fingerprint=ef096b394c21f6b8f5394a9f55828ea14dff308dba00041ef0232d6d85b63488 I0216 18:13:15.509380 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 8.828691ms to process 1 records I0216 18:13:15.509464 1 tasks_processing.go:71] worker 8 working on feature_gates task. I0216 18:13:15.509485 1 recorder.go:75] Recording config/proxy with fingerprint=31fe3984b6f2d8997e87452fe6f57b5afb43efa62b71d22dee7856d8dc2ee4c7 I0216 18:13:15.509494 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 9.32065ms to process 1 records I0216 18:13:15.510200 1 tasks_processing.go:71] worker 7 working on validating_webhook_configurations task. I0216 18:13:15.510278 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=f4eed953c97f4985b1fc9cac176b870c91a3ddfeb81663405d50d47d8393b2dc I0216 18:13:15.510294 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 10.294217ms to process 1 records I0216 18:13:15.521103 1 tasks_processing.go:71] worker 29 working on openstack_version task. E0216 18:13:15.521324 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0216 18:13:15.521345 1 gather.go:177] gatherer "clusterconfig" function "machines" took 18.086214ms to process 0 records I0216 18:13:15.521354 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 17.121105ms to process 0 records I0216 18:13:15.521360 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 16.630381ms to process 0 records I0216 18:13:15.521365 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 17.425866ms to process 0 records I0216 18:13:15.521389 1 tasks_processing.go:71] worker 12 working on cluster_apiserver task. I0216 18:13:15.521434 1 tasks_processing.go:71] worker 27 working on lokistack task. I0216 18:13:15.521531 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=443fc5c0fec31a30af4cbbcb7fb3b75efc3fb79ed1d6773b7def94481470a967 I0216 18:13:15.521636 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=4f5baae6de613a363a83bdd1864ef53e61f38f1fe5a72af35f7791cae9a153df I0216 18:13:15.521721 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 20.802021ms to process 2 records I0216 18:13:15.521546 1 tasks_processing.go:71] worker 10 working on ingress_certificates task. I0216 18:13:15.522051 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=6331f73fe2e5585eaa3c6414d59d2a821947f3979507562359e0d4ca0cbe109c I0216 18:13:15.522066 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 20.289245ms to process 1 records I0216 18:13:15.522076 1 tasks_processing.go:71] worker 21 working on olm_operators task. I0216 18:13:15.522151 1 gather_logs.go:145] no pods in namespace were found E0216 18:13:15.522166 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0216 18:13:15.522215 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 18.817724ms to process 0 records I0216 18:13:15.522224 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 21.194202ms to process 0 records I0216 18:13:15.521563 1 tasks_processing.go:71] worker 20 working on image task. I0216 18:13:15.522294 1 tasks_processing.go:71] worker 14 working on install_plans task. I0216 18:13:15.522350 1 tasks_processing.go:71] worker 25 working on active_alerts task. W0216 18:13:15.522446 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0216 18:13:15.522487 1 tasks_processing.go:71] worker 19 working on networks task. I0216 18:13:15.522536 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0216 18:13:15.522587 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 20.128946ms to process 1 records I0216 18:13:15.522619 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 110.599µs to process 0 records I0216 18:13:15.522648 1 tasks_processing.go:71] worker 25 working on nodes task. I0216 18:13:15.528832 1 tasks_processing.go:71] worker 26 working on config_maps task. I0216 18:13:15.529069 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 19.771814ms to process 0 records I0216 18:13:15.530307 1 tasks_processing.go:71] worker 3 working on openstack_controlplanes task. I0216 18:13:15.531012 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s W0216 18:13:15.531211 1 operator.go:288] started I0216 18:13:15.531214 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0216 18:13:15.531483 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0216 18:13:15.531623 1 recorder.go:75] Recording config/infrastructure with fingerprint=6b86eab341545e68f51b81b45ec87c063d431817e84945f5ba5ad05ce234fbd4 I0216 18:13:15.531695 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 22.886075ms to process 1 records I0216 18:13:15.531753 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 24.930625ms to process 0 records I0216 18:13:15.531766 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 25.318629ms to process 0 records E0216 18:13:15.531783 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope W0216 18:13:15.531724 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0216 18:13:15.531793 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 25.417554ms to process 0 records I0216 18:13:15.531820 1 tasks_processing.go:71] worker 9 working on container_images task. I0216 18:13:15.532268 1 recorder.go:75] Recording config/authentication with fingerprint=fd203e6bfa329e73296e0f3c0b077b2686a19f7ae3fe4838a78a3b37fbdea6f6 I0216 18:13:15.532304 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 23.119028ms to process 1 records I0216 18:13:15.532315 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 25.17875ms to process 0 records I0216 18:13:15.532322 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 27.294065ms to process 0 records I0216 18:13:15.532497 1 tasks_processing.go:71] worker 11 working on pdbs task. I0216 18:13:15.532594 1 tasks_processing.go:71] worker 31 working on ceph_cluster task. I0216 18:13:15.532698 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=bda7473463bc76084133db5c87a7e3e511c94121fd3596ac8a9ad673d4f22177 I0216 18:13:15.532735 1 tasks_processing.go:71] worker 13 working on nodenetworkstates task. I0216 18:13:15.532769 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=a4c8600357123a0901f26a487dca3e64cb3fffac5d50120fe846088612619a82 I0216 18:13:15.532820 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=bfaa38149094294b0cdba33a119a5dae923d356c84e8341fb758970c2f85cce3 I0216 18:13:15.532829 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 27.209421ms to process 3 records I0216 18:13:15.532846 1 tasks_processing.go:71] worker 28 working on oauths task. I0216 18:13:15.532942 1 tasks_processing.go:71] worker 6 working on operators_pods_and_events task. I0216 18:13:15.533014 1 tasks_processing.go:71] worker 1 working on sap_pods task. I0216 18:13:15.533024 1 tasks_processing.go:74] worker 16 stopped. I0216 18:13:15.533864 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=7702f2caff49483046b4c6236c827cedb1b13863c8cb2be456e6560f825c0efb I0216 18:13:15.533892 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 29.398446ms to process 1 records I0216 18:13:15.535191 1 tasks_processing.go:74] worker 27 stopped. I0216 18:13:15.535248 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 13.737068ms to process 0 records I0216 18:13:15.535306 1 tasks_processing.go:74] worker 29 stopped. I0216 18:13:15.535333 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 14.180859ms to process 0 records I0216 18:13:15.535377 1 tasks_processing.go:74] worker 8 stopped. I0216 18:13:15.535512 1 recorder.go:75] Recording config/featuregate with fingerprint=d10b9d1e1584773398878c7bdf335cb1f29dd176b9e67a1f6181ab6861c791a5 I0216 18:13:15.535530 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 25.896219ms to process 1 records I0216 18:13:15.536533 1 tasks_processing.go:74] worker 30 stopped. I0216 18:13:15.536598 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 32.635823ms to process 0 records I0216 18:13:15.536769 1 tasks_processing.go:74] worker 3 stopped. I0216 18:13:15.536793 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 6.399988ms to process 0 records I0216 18:13:15.536998 1 tasks_processing.go:74] worker 20 stopped. I0216 18:13:15.537140 1 recorder.go:75] Recording config/image with fingerprint=ac6cbead975ba18091ae6aadcf3d538711f49243e4becc7cef7a2379155fa69f I0216 18:13:15.537193 1 gather.go:177] gatherer "clusterconfig" function "image" took 14.728159ms to process 1 records I0216 18:13:15.537354 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=8e7c526b19febff8d1d7a74b022d87153f6f5b7d80069880739d563983aadc15 I0216 18:13:15.537358 1 tasks_processing.go:74] worker 7 stopped. I0216 18:13:15.537520 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=3fc7461153dba9899566998b3ef28b443daeef7fb90ba22495f973d523dfad64 I0216 18:13:15.537560 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=38da963e0a0e88b08aa5d8c3e4b4608239f7badd87111fa53201ee3c4a73b740 I0216 18:13:15.537615 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=d04468cecb609cb9d90ff2d45f2e55401267ab9fac21e718884d5f2716bf39eb I0216 18:13:15.537742 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=9bee3657ad4d057c082331c101bdfa622d998521b194b6a1c8f8d49adcf865b6 I0216 18:13:15.537829 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=952929e8749e2cb73400d0df7faba64545115b53cacc3eff391e38623aa4881a I0216 18:13:15.537895 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=332bc5703fbe4775fb1b776ddd01c9ca7a9d9d30793e8d5adafdfeeee0846952 I0216 18:13:15.537990 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=7028bf58b043949e0b98f3fc938517d4121bcd49ff14602c88a1bdb1ebcc9820 I0216 18:13:15.538055 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=47a29a77f889b75159bb8f837b0271150f17538c70111a486c9fcedca6f82e22 I0216 18:13:15.538140 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=b37d64a6403992ac76b1cf591312ad0a98c46b8f7e987bb11b9d95ba1fe75955 I0216 18:13:15.538182 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=26a9d9ef0552beaa09bf95c2e1c47fa7375d20877b0c6e7460f3947fc96004d9 I0216 18:13:15.538196 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 26.903649ms to process 11 records I0216 18:13:15.545566 1 tasks_processing.go:74] worker 31 stopped. I0216 18:13:15.545587 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 12.952466ms to process 0 records I0216 18:13:15.545617 1 tasks_processing.go:74] worker 13 stopped. I0216 18:13:15.545625 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 12.873969ms to process 0 records I0216 18:13:15.546085 1 tasks_processing.go:74] worker 12 stopped. I0216 18:13:15.546139 1 controller.go:212] Source scaController *sca.Controller is not ready I0216 18:13:15.546155 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0216 18:13:15.546162 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0216 18:13:15.546166 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0216 18:13:15.546170 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0216 18:13:15.546206 1 controller.go:489] The operator is still being initialized I0216 18:13:15.546218 1 controller.go:512] The operator is healthy I0216 18:13:15.546361 1 recorder.go:75] Recording config/apiserver with fingerprint=afa270c03515d2c5e81339cec3bc90abcfcdf8d09f1d839281b36fbe6d9f3416 I0216 18:13:15.546410 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 24.67487ms to process 1 records I0216 18:13:15.546555 1 tasks_processing.go:74] worker 28 stopped. I0216 18:13:15.546870 1 recorder.go:75] Recording config/oauth with fingerprint=b3aa4934ebda8b227bc658ead25e8a1c9cc6609f02f7c0f419bf4a16b0de87a3 I0216 18:13:15.546928 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 13.300047ms to process 1 records I0216 18:13:15.546952 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 45.262369ms to process 0 records I0216 18:13:15.546984 1 tasks_processing.go:74] worker 15 stopped. I0216 18:13:15.547073 1 tasks_processing.go:74] worker 19 stopped. I0216 18:13:15.547211 1 recorder.go:75] Recording config/network with fingerprint=38ec3b8cebe91814f6d8fb096bf6fd113bba0affeed2b10fe110cb7189dd0a2b I0216 18:13:15.547271 1 gather.go:177] gatherer "clusterconfig" function "networks" took 24.082085ms to process 1 records I0216 18:13:15.547429 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=4e3bffab83fdcf77c4150d021527055b5bfd64c78bf07bf9f982dfcb6c8be1d9 I0216 18:13:15.547459 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=41590ddca304b876049540d0e712b16e2ad416045ff9f2791ee6a660e7915685 I0216 18:13:15.547458 1 tasks_processing.go:74] worker 11 stopped. I0216 18:13:15.547481 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=c8c9e3260f6385ea91a3120676552405f5c9829c8508a2e92502e1df03a198db I0216 18:13:15.547491 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 14.359654ms to process 3 records I0216 18:13:15.547576 1 tasks_processing.go:74] worker 23 stopped. I0216 18:13:15.548242 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=864731cfb6cb027c24dd8a627f5836baa743b09ea1cd5e808bdb73db136e8b94 I0216 18:13:15.548519 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=4b9e5cf4d639c43400ab1cbd7fc0e329f9bd7c12cad83614a1b89c462b67c47a I0216 18:13:15.548538 1 gather.go:177] gatherer "clusterconfig" function "crds" took 43.760877ms to process 2 records I0216 18:13:15.551508 1 tasks_processing.go:74] worker 1 stopped. I0216 18:13:15.551533 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 18.481724ms to process 0 records I0216 18:13:15.551833 1 tasks_processing.go:74] worker 5 stopped. I0216 18:13:15.551898 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 50.447879ms to process 0 records I0216 18:13:15.553317 1 prometheus_rules.go:88] Prometheus rules successfully created I0216 18:13:15.553712 1 tasks_processing.go:74] worker 18 stopped. I0216 18:13:15.553998 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=16cd07ad95a64b258e31e46b467c61f96babf9dac423849eac081462ef8a470e I0216 18:13:15.554157 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=1057d5b9c73df675f63920b04766ffaec4981fe614ea3ea08ea024550d36bcab I0216 18:13:15.554206 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 44.971408ms to process 2 records I0216 18:13:15.554219 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 49.48855ms to process 0 records I0216 18:13:15.554228 1 tasks_processing.go:74] worker 17 stopped. I0216 18:13:15.555804 1 tasks_processing.go:74] worker 21 stopped. I0216 18:13:15.555883 1 recorder.go:75] Recording config/olm_operators with fingerprint=1614482ef31b0b00ebf60aea6ac0bd3f762ea536b474c2282ffc752723fb6d90 I0216 18:13:15.555901 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 33.716905ms to process 1 records E0216 18:13:15.559037 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2717e28ab1-5431-4310-a2bb-bd9f61798466%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.9:39700->172.30.0.10:53: read: connection refused I0216 18:13:15.559054 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2717e28ab1-5431-4310-a2bb-bd9f61798466%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.9:39700->172.30.0.10:53: read: connection refused I0216 18:13:15.564135 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0216 18:13:15.564213 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0216 18:13:15.564706 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0216 18:13:15.570436 1 base_controller.go:82] Caches are synced for ConfigController I0216 18:13:15.570450 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0216 18:13:15.572588 1 tasks_processing.go:74] worker 10 stopped. E0216 18:13:15.572609 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0216 18:13:15.572620 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2oggm30vmofomplkfldgt27943ql796q-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2oggm30vmofomplkfldgt27943ql796q-primary-cert-bundle-secret" not found I0216 18:13:15.572730 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=51348f12bbb984ab3cfd542c660ae81953105f7685ef3b58ff09122e15a34592 I0216 18:13:15.572752 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 50.781282ms to process 1 records I0216 18:13:15.574487 1 tasks_processing.go:74] worker 26 stopped. E0216 18:13:15.574506 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0216 18:13:15.574514 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0216 18:13:15.574519 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0216 18:13:15.574534 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0216 18:13:15.574560 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0216 18:13:15.574570 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0216 18:13:15.574576 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0216 18:13:15.574582 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0216 18:13:15.574627 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0216 18:13:15.574709 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0216 18:13:15.574720 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 45.601126ms to process 7 records I0216 18:13:15.576141 1 configmapobserver.go:84] configmaps "insights-config" not found I0216 18:13:15.617715 1 tasks_processing.go:74] worker 4 stopped. I0216 18:13:15.618082 1 recorder.go:75] Recording config/version with fingerprint=95e1af8ed05466ac5e01d187cc2c9767d48cc9a9f1f982dab69dc924750816e4 I0216 18:13:15.618154 1 recorder.go:75] Recording config/id with fingerprint=fecc0832a6735d1248045cfa741194359954eaa18b1c5d670103f29f37391a10 I0216 18:13:15.618171 1 gather.go:177] gatherer "clusterconfig" function "version" took 108.838805ms to process 2 records I0216 18:13:15.627923 1 tasks_processing.go:74] worker 9 stopped. I0216 18:13:15.630220 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-smtwj with fingerprint=96766512f49454697ae095feaf723bc8b27ea68da8c6773eb945537d57c7b150 I0216 18:13:15.630757 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-p4f8s with fingerprint=dccf86ee2802802d325d962201a9b55250b13105ae93447be7e1186ca45691f5 I0216 18:13:15.630874 1 recorder.go:75] Recording config/running_containers with fingerprint=7f361651e53eec4086c77b7c6cbd1068ca21b2f5a39a3a5601364520d8dc82e0 I0216 18:13:15.630910 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 96.068592ms to process 3 records I0216 18:13:15.631038 1 tasks_processing.go:74] worker 25 stopped. I0216 18:13:15.631303 1 recorder.go:75] Recording config/node/ip-10-0-1-84.ec2.internal with fingerprint=215592531367902116020501eb697148b6e81f94ba7fdbeae9fea06c493fd86d I0216 18:13:15.631394 1 recorder.go:75] Recording config/node/ip-10-0-2-109.ec2.internal with fingerprint=b1b7f42ecd08124cfd8168c69af95c54d38d62b49572e5b33664e0cf2b86473d I0216 18:13:15.631405 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 106.246746ms to process 2 records I0216 18:13:15.631625 1 base_controller.go:82] Caches are synced for LoggingSyncer I0216 18:13:15.631651 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0216 18:13:15.631685 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0216 18:13:15.631698 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 128.312345ms to process 1 records I0216 18:13:15.631631 1 tasks_processing.go:74] worker 24 stopped. I0216 18:13:15.640755 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0216 18:13:15.648450 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.9:49435->172.30.0.10:53: read: connection refused I0216 18:13:15.648472 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.9:49435->172.30.0.10:53: read: connection refused W0216 18:13:16.526306 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0216 18:13:16.766275 1 gather_cluster_operator_pods_and_events.go:121] Found 16 pods with 19 containers I0216 18:13:16.766295 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1324517 bytes I0216 18:13:16.766792 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-jg7sl pod in namespace openshift-dns (previous: false). I0216 18:13:16.995794 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-jg7sl pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-jg7sl\" is waiting to start: ContainerCreating" I0216 18:13:16.995814 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-jg7sl\" is waiting to start: ContainerCreating" I0216 18:13:16.995822 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-jg7sl pod in namespace openshift-dns (previous: false). I0216 18:13:17.166915 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0216 18:13:17.169573 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-jg7sl pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-jg7sl\" is waiting to start: ContainerCreating" I0216 18:13:17.169591 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-jg7sl\" is waiting to start: ContainerCreating" I0216 18:13:17.169618 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-kr85l pod in namespace openshift-dns (previous: false). I0216 18:13:17.392040 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kr85l pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-kr85l\" is waiting to start: ContainerCreating" I0216 18:13:17.392059 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-kr85l\" is waiting to start: ContainerCreating" I0216 18:13:17.392069 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-kr85l pod in namespace openshift-dns (previous: false). W0216 18:13:17.527207 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0216 18:13:17.568310 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-kr85l pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-kr85l\" is waiting to start: ContainerCreating" I0216 18:13:17.568330 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-kr85l\" is waiting to start: ContainerCreating" I0216 18:13:17.568342 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-ccgql pod in namespace openshift-dns (previous: false). I0216 18:13:17.768824 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0216 18:13:17.768846 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-jtmnx pod in namespace openshift-dns (previous: false). I0216 18:13:17.770012 1 tasks_processing.go:74] worker 2 stopped. I0216 18:13:17.770062 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=9d50871c7be441688a926ceb2c28e7ad17f516db3cde22e038bc0f58343493b6 I0216 18:13:17.770098 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=34e630c2d35f99c1230d123c36b0c8462b04734eabd40b682027d283281b985b I0216 18:13:17.770128 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0216 18:13:17.770156 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=c944dc9c80fd645d5dd69fd6479abe98e8a019b0d22c5ac098bff586da3280ea I0216 18:13:17.770190 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0216 18:13:17.770213 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=eac69ef94030117e32c211eec8529d8d593804ed2d1fd5538738e84211f80de0 I0216 18:13:17.770243 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=36fdaddb3670528c66c031e4f57ac9d4a012aa08ca08ba3561821a78c42ef084 I0216 18:13:17.770268 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=15d2eb4a80ab9eeb0567da1d35bc5b795746f037af18a1ded97ce8083d5dd42f I0216 18:13:17.770299 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=e0030f23bb33bee37725a6b35a4271bd55b7bd8adc801dedac707e401fc0b96c I0216 18:13:17.770308 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e5ff11d57817f84a678f6fa9565af55bd1120227c16a21933637ab62675a6d70 I0216 18:13:17.770324 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=0563512c09724f2d518aaacd6b210c2fd2b18f162f6ad7c64cf2c9cf3e8236d0 I0216 18:13:17.770334 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0216 18:13:17.770350 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=89f036461167bcbf85bb70187b512f89a025e83ca1b5363ca05fcbe4f2c4ddc1 I0216 18:13:17.770360 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0216 18:13:17.770375 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=0382a5d00e37795bc4dfef1337471a4219639a7c0ea70182e75ee26bb5e3671c I0216 18:13:17.770385 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0216 18:13:17.770404 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=ea5cb65698bbb3b0c3e75b992124506cdeb6bb325b80c08127c1dc1abc1d62fc I0216 18:13:17.770414 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0216 18:13:17.770430 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=22d339e80b7b4d105915825f393410c5686bd220c5ab5a2c7a6d4a9a08f9d7e2 I0216 18:13:17.770586 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=651f8b58e1ebcb62d64aba04bc4216f00ca60db9450822f888fc7b848a1b24a7 I0216 18:13:17.770597 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0216 18:13:17.770605 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0216 18:13:17.770633 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0216 18:13:17.770655 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=53639c2abf57893d4972d67d7e35aa235427f3b4d90b67d613407cf8cd166137 I0216 18:13:17.770712 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=b19be015df66af03ee7fe2b28702f6a8e683f939bdd31b695054e0eb58610888 I0216 18:13:17.770723 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0216 18:13:17.770740 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=7fa65c96ff60c34feb9970392b4d1b35775b04dd7c6d55244e4d5ebb59aefc98 I0216 18:13:17.770750 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0216 18:13:17.770763 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=cc94c14d5f5636d6d8be2076c6009ca6b2d63fbc29d40c8748b9db59924b0839 I0216 18:13:17.770778 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=1287fa79f5026ad4ed98afc71a202e650c17d58f20beb51956fa43986216c6e9 I0216 18:13:17.770793 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=e078231d298d86a822cd0be602dc78b5a9c20e93549848f3194ee0c5b810ca4b I0216 18:13:17.770808 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=a11491cd3b635572e4d185bb083906b832b98c7da3ae18008e8392e09288995b I0216 18:13:17.770831 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=37f0e3b3282d9fd0c14244ba3880805a37275f6150ffefed29482742dbe9aaeb I0216 18:13:17.770846 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0216 18:13:17.770876 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=07b0798807567905b78da9a23862935aa5988ba1a576d3b15add1a79cde6c169 I0216 18:13:17.770894 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0216 18:13:17.770902 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0216 18:13:17.770909 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.266625764s to process 37 records I0216 18:13:17.969479 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0216 18:13:17.969542 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-58c6774647-lmtxb pod in namespace openshift-image-registry (previous: false). I0216 18:13:18.170395 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-58c6774647-lmtxb pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-58c6774647-lmtxb\" is waiting to start: ContainerCreating" I0216 18:13:18.170418 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-58c6774647-lmtxb\" is waiting to start: ContainerCreating" I0216 18:13:18.170463 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-58c6774647-vzcnt pod in namespace openshift-image-registry (previous: false). I0216 18:13:18.370266 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-58c6774647-vzcnt pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-58c6774647-vzcnt\" is waiting to start: ContainerCreating" I0216 18:13:18.370292 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-58c6774647-vzcnt\" is waiting to start: ContainerCreating" I0216 18:13:18.370384 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-685cdb9d4d-xxf8l pod in namespace openshift-image-registry (previous: false). W0216 18:13:18.527324 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0216 18:13:18.562380 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0216 18:13:18.562402 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-8nvmd pod in namespace openshift-image-registry (previous: false). I0216 18:13:18.769596 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0216 18:13:18.769616 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-p9gxs pod in namespace openshift-image-registry (previous: false). I0216 18:13:18.968859 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0216 18:13:18.968888 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-56bf9967bf-j54ts pod in namespace openshift-ingress (previous: false). I0216 18:13:19.169521 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-56bf9967bf-j54ts pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-56bf9967bf-j54ts\" is waiting to start: ContainerCreating" I0216 18:13:19.169542 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-56bf9967bf-j54ts\" is waiting to start: ContainerCreating" I0216 18:13:19.169556 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-56bf9967bf-s6rgl pod in namespace openshift-ingress (previous: false). I0216 18:13:19.369530 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-56bf9967bf-s6rgl pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-56bf9967bf-s6rgl\" is waiting to start: ContainerCreating" I0216 18:13:19.369557 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-56bf9967bf-s6rgl\" is waiting to start: ContainerCreating" I0216 18:13:19.369568 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7548757bc-hgtwt pod in namespace openshift-ingress (previous: false). W0216 18:13:19.527007 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0216 18:13:19.568921 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7548757bc-hgtwt pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7548757bc-hgtwt\" is waiting to start: ContainerCreating" I0216 18:13:19.568944 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7548757bc-hgtwt\" is waiting to start: ContainerCreating" I0216 18:13:19.568974 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-28wrb pod in namespace openshift-ingress-canary (previous: false). I0216 18:13:19.768371 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-28wrb pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-28wrb\" is waiting to start: ContainerCreating" I0216 18:13:19.768390 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-28wrb\" is waiting to start: ContainerCreating" I0216 18:13:19.768419 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-z95xg pod in namespace openshift-ingress-canary (previous: false). I0216 18:13:19.969434 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-z95xg pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-z95xg\" is waiting to start: ContainerCreating" I0216 18:13:19.969453 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-z95xg\" is waiting to start: ContainerCreating" I0216 18:13:19.969466 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-ct579 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0216 18:13:20.170625 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-ct579 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0216 18:13:20.375878 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-v95bc pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). W0216 18:13:20.523255 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0216 18:13:20.523283 1 tasks_processing.go:74] worker 22 stopped. E0216 18:13:20.523294 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0216 18:13:20.523305 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0216 18:13:20.523322 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0216 18:13:20.523331 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.022319077s to process 1 records I0216 18:13:20.572132 1 tasks_processing.go:74] worker 6 stopped. I0216 18:13:20.572278 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=635a36f90573c2f88856acb720cac349818199ae8d1b31017b16485d474fe3db I0216 18:13:20.572317 1 recorder.go:75] Recording events/openshift-dns with fingerprint=aba38facc325958e0ff3055cb7dd735a207dec1ce843cf4ff288838d3b7cb598 I0216 18:13:20.572391 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=d80c3836a9a20d2fe07d09179df57359a58aacc770b31b387af082681e84866d I0216 18:13:20.572419 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=2fd30da6daf15cf2bd25735ccb58c789730815a8cbe87dbcfab2281b9fa547ef I0216 18:13:20.572464 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=0b807c098de9982296cf849186fb73c18b8b65f493fd782cf96668a00cb565f2 I0216 18:13:20.572478 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=d045776088fa4fe6a9f8275d679f8af84573f40444bd76662e0f3727d5ebe6c6 I0216 18:13:20.572501 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=299a9e120d8246e1bee74c72c27eca180a6fff410a7871a53660fe55291e41c6 I0216 18:13:20.572805 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=bd6d130cac2e30ce7057a658cf31b38699f89aa25f39d76d3886911c0d8d5899 I0216 18:13:20.573259 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-jg7sl with fingerprint=0878282d2c8f418b27acd31fa30341d06609fefc410b03b5b0369912f94f323f I0216 18:13:20.573503 1 recorder.go:75] Recording config/pod/openshift-dns/dns-default-kr85l with fingerprint=4159f7fa8c2dbbe09cb4e50ca8e5d7b9456b6c0a028b6832f608e7fdebc77c7c I0216 18:13:20.573738 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-58c6774647-lmtxb with fingerprint=be18e930640b38ed29bce4e05860060b64e6df2df6cbf7145e5cba9394def614 I0216 18:13:20.573962 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-58c6774647-vzcnt with fingerprint=4fd5223c117f9c33d055f2e934aaa5b2739a9e14445ae341c859b83b3cc09ec1 I0216 18:13:20.574133 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-685cdb9d4d-xxf8l with fingerprint=80e65e0bf1ed13b71e04bca6e6a68d7a4ee171c7d57adf47ef5f262f47d6d683 I0216 18:13:20.574258 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-28wrb with fingerprint=c6ecc205e589b1498e9fea66a823ca1f0d096ee8472da5172b6f2bc5dd64a770 I0216 18:13:20.574415 1 recorder.go:75] Recording config/pod/openshift-ingress-canary/ingress-canary-z95xg with fingerprint=33dd3dc5ce94a8e88721af31e112182f39523a90c3aaf1e41c4908ccb7343c97 I0216 18:13:20.574460 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-ct579/migrator_current.log with fingerprint=32493b36dffe33e3d019dd3b52eed7634e8b7deb43969ee169f913ca71bfca37 I0216 18:13:20.574485 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-ct579/graceful-termination_current.log with fingerprint=f1015d449bd645f99c06eb99c8c9c0aee3d1c69f34430172e4eace08388a190e I0216 18:13:20.574606 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-v95bc/kube-storage-version-migrator-operator_current.log with fingerprint=3f8ab667e96378c12b97e758d14cee02768c4e72951892c223a4a8243e4ab7b4 I0216 18:13:20.574633 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 5.039167805s to process 18 records I0216 18:13:26.041726 1 configmapobserver.go:84] configmaps "insights-config" not found I0216 18:13:28.027742 1 tasks_processing.go:74] worker 14 stopped. I0216 18:13:28.027805 1 recorder.go:75] Recording config/installplans with fingerprint=b6ae0e2549358513c087729c711e8e1ad6f2144adc0ffa716b1a475ed1e6ddde I0216 18:13:28.027824 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.505399287s to process 1 records I0216 18:13:28.713013 1 tasks_processing.go:74] worker 0 stopped. I0216 18:13:28.713374 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=821c279aa16c8e4698dc922d33710442ae347dfdb57a77b1ae98de8edf882f8f I0216 18:13:28.713392 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.207129756s to process 1 records E0216 18:13:28.713448 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.213s with: function \"support_secret\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"ingress_certificates\" failed with an error, function \"config_maps\" failed with an error, function \"dvo_metrics\" failed with an error" I0216 18:13:28.714565 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "support_secret" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "ingress_certificates" failed with an error, function "config_maps" failed with an error, function "dvo_metrics" failed with an error I0216 18:13:28.714579 1 periodic.go:209] Running workloads gatherer I0216 18:13:28.714595 1 tasks_processing.go:45] number of workers: 2 I0216 18:13:28.714601 1 tasks_processing.go:69] worker 1 listening for tasks. I0216 18:13:28.714605 1 tasks_processing.go:71] worker 1 working on workload_info task. I0216 18:13:28.714615 1 tasks_processing.go:69] worker 0 listening for tasks. I0216 18:13:28.714632 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0216 18:13:28.739779 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 20s for image data I0216 18:13:28.746176 1 tasks_processing.go:74] worker 0 stopped. I0216 18:13:28.746198 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 31.526052ms to process 0 records I0216 18:13:28.750374 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (11ms) I0216 18:13:28.761023 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (11ms) I0216 18:13:28.771576 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (11ms) I0216 18:13:28.784121 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (13ms) I0216 18:13:28.794066 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (10ms) I0216 18:13:28.804538 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (10ms) I0216 18:13:28.814416 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (10ms) I0216 18:13:28.824470 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (10ms) I0216 18:13:28.836195 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (12ms) I0216 18:13:28.846245 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (10ms) I0216 18:13:28.856164 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (10ms) I0216 18:13:28.950572 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (94ms) I0216 18:13:29.051791 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (101ms) I0216 18:13:29.151176 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (99ms) I0216 18:13:29.250255 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (99ms) I0216 18:13:29.350857 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (101ms) I0216 18:13:29.451305 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0216 18:13:29.551433 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0216 18:13:29.651099 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0216 18:13:29.750580 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (99ms) I0216 18:13:29.851001 1 gather_workloads_info.go:387] No image sha256:f82357030795138d2081ecc5172092222b0f4faea27e9a7a0474fbeae29111ad (100ms) I0216 18:13:29.952866 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (102ms) I0216 18:13:30.053898 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0216 18:13:30.151765 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (98ms) I0216 18:13:30.252502 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (101ms) I0216 18:13:30.350993 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (98ms) I0216 18:13:30.389644 1 configmapobserver.go:84] configmaps "insights-config" not found I0216 18:13:30.451510 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0216 18:13:30.552024 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (100ms) I0216 18:13:30.590626 1 configmapobserver.go:84] configmaps "insights-config" not found I0216 18:13:30.651245 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (99ms) I0216 18:13:30.751688 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (100ms) I0216 18:13:30.851474 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0216 18:13:30.950867 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0216 18:13:30.950908 1 tasks_processing.go:74] worker 1 stopped. E0216 18:13:30.950920 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0216 18:13:30.951168 1 recorder.go:75] Recording config/workload_info with fingerprint=6c766336a5d05a747ac41e155109e587d57a70b800e05463abbfca9ee7ae7933 I0216 18:13:30.951188 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.236294004s to process 1 records E0216 18:13:30.951220 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.236s with: function \"workload_info\" failed with an error" I0216 18:13:30.952338 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0216 18:13:30.952355 1 periodic.go:209] Running conditional gatherer I0216 18:13:30.958590 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0216 18:13:30.966027 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.9:39086->172.30.0.10:53: read: connection refused E0216 18:13:30.966302 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0216 18:13:30.966384 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0216 18:13:30.972991 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0216 18:13:30.973010 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973015 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973019 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973022 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973025 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973028 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973031 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973034 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0216 18:13:30.973036 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0216 18:13:30.973056 1 tasks_processing.go:45] number of workers: 3 I0216 18:13:30.973071 1 tasks_processing.go:69] worker 2 listening for tasks. I0216 18:13:30.973076 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0216 18:13:30.973083 1 tasks_processing.go:69] worker 0 listening for tasks. I0216 18:13:30.973094 1 tasks_processing.go:69] worker 1 listening for tasks. I0216 18:13:30.973114 1 tasks_processing.go:74] worker 1 stopped. I0216 18:13:30.973114 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0216 18:13:30.973145 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0216 18:13:30.973205 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0216 18:13:30.973240 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 964ns to process 1 records I0216 18:13:30.973279 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0216 18:13:30.973319 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.237µs to process 1 records I0216 18:13:30.973325 1 tasks_processing.go:74] worker 0 stopped. I0216 18:13:30.973444 1 tasks_processing.go:74] worker 2 stopped. I0216 18:13:30.973460 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 280.286µs to process 0 records I0216 18:13:30.973486 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.9:39086->172.30.0.10:53: read: connection refused I0216 18:13:30.973503 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0216 18:13:30.999083 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=6450177c3358c2c91b082f360aeadab3e6df09c18105df3b1d9a069fca4ab734 I0216 18:13:30.999229 1 diskrecorder.go:70] Writing 115 records to /var/lib/insights-operator/insights-2026-02-16-181330.tar.gz I0216 18:13:31.006856 1 diskrecorder.go:51] Wrote 115 records to disk in 7ms I0216 18:13:31.006905 1 periodic.go:278] Gathering cluster info every 2h0m0s I0216 18:13:31.006930 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0216 18:13:42.303873 1 configmapobserver.go:84] configmaps "insights-config" not found I0216 18:14:24.907381 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="fc14240589b4414bd69cebccda3d35d2f2d1b5881618e9d7322d3afdf8b407df") W0216 18:14:24.907420 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0216 18:14:24.907481 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="864d3431c6f1ef397a8612f7d6955def2363c77ac2c4b1851c7ece50b250d8f5") I0216 18:14:24.907490 1 base_controller.go:181] Shutting down ConfigController ... I0216 18:14:24.907500 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0216 18:14:24.907510 1 periodic.go:170] Shutting down I0216 18:14:24.907520 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0216 18:14:24.907795 1 base_controller.go:113] All ConfigController workers have been terminated I0216 18:14:24.907533 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0216 18:14:24.907801 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain"