W0129 16:51:26.823965 1 cmd.go:257] Using insecure, self-signed certificates I0129 16:51:27.339368 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0129 16:51:27.339680 1 observer_polling.go:159] Starting file observer I0129 16:51:28.171266 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0129 16:51:28.171520 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0129 16:51:28.173139 1 secure_serving.go:57] Forcing use of http/1.1 only W0129 16:51:28.173233 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0129 16:51:28.173262 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0129 16:51:28.173286 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0129 16:51:28.173314 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0129 16:51:28.173335 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0129 16:51:28.173357 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0129 16:51:28.174096 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0129 16:51:28.179370 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0129 16:51:28.179387 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"b6e3fd3b-f632-471c-8898-e97432ba9ac9", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0129 16:51:28.184094 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0129 16:51:28.184116 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0129 16:51:28.184110 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0129 16:51:28.184114 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0129 16:51:28.184142 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0129 16:51:28.184155 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0129 16:51:28.184398 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-1927861705/tls.crt::/tmp/serving-cert-1927861705/tls.key" I0129 16:51:28.184718 1 secure_serving.go:213] Serving securely on [::]:8443 I0129 16:51:28.184748 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0129 16:51:28.192098 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0129 16:51:28.192124 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0129 16:51:28.192211 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0129 16:51:28.207695 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0129 16:51:28.207713 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0129 16:51:28.214211 1 secretconfigobserver.go:119] support secret does not exist I0129 16:51:28.222752 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0129 16:51:28.227974 1 secretconfigobserver.go:119] support secret does not exist I0129 16:51:28.235226 1 recorder.go:161] Pruning old reports every 4h46m17s, max age is 288h0m0s I0129 16:51:28.244845 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0129 16:51:28.244850 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0129 16:51:28.244861 1 insightsreport.go:296] Starting report retriever I0129 16:51:28.244864 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0129 16:51:28.244867 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0129 16:51:28.244906 1 periodic.go:209] Running clusterconfig gatherer I0129 16:51:28.244974 1 tasks_processing.go:45] number of workers: 32 I0129 16:51:28.245004 1 tasks_processing.go:69] worker 31 listening for tasks. I0129 16:51:28.245008 1 tasks_processing.go:71] worker 31 working on oauths task. I0129 16:51:28.245041 1 tasks_processing.go:69] worker 15 listening for tasks. I0129 16:51:28.245051 1 tasks_processing.go:71] worker 15 working on openstack_controlplanes task. I0129 16:51:28.245056 1 tasks_processing.go:69] worker 23 listening for tasks. I0129 16:51:28.245061 1 tasks_processing.go:69] worker 19 listening for tasks. I0129 16:51:28.245070 1 tasks_processing.go:69] worker 16 listening for tasks. I0129 16:51:28.245070 1 tasks_processing.go:69] worker 27 listening for tasks. I0129 16:51:28.245075 1 tasks_processing.go:69] worker 20 listening for tasks. I0129 16:51:28.245080 1 tasks_processing.go:69] worker 17 listening for tasks. I0129 16:51:28.245081 1 tasks_processing.go:69] worker 24 listening for tasks. I0129 16:51:28.245084 1 tasks_processing.go:69] worker 22 listening for tasks. I0129 16:51:28.245088 1 tasks_processing.go:69] worker 25 listening for tasks. I0129 16:51:28.245092 1 tasks_processing.go:69] worker 18 listening for tasks. I0129 16:51:28.245095 1 tasks_processing.go:69] worker 29 listening for tasks. I0129 16:51:28.245099 1 tasks_processing.go:69] worker 26 listening for tasks. I0129 16:51:28.245096 1 tasks_processing.go:69] worker 21 listening for tasks. I0129 16:51:28.245101 1 tasks_processing.go:69] worker 28 listening for tasks. I0129 16:51:28.245111 1 tasks_processing.go:69] worker 0 listening for tasks. I0129 16:51:28.245104 1 tasks_processing.go:69] worker 7 listening for tasks. I0129 16:51:28.245110 1 tasks_processing.go:69] worker 9 listening for tasks. I0129 16:51:28.245116 1 tasks_processing.go:69] worker 30 listening for tasks. I0129 16:51:28.245120 1 tasks_processing.go:69] worker 8 listening for tasks. I0129 16:51:28.245120 1 tasks_processing.go:69] worker 1 listening for tasks. I0129 16:51:28.245126 1 tasks_processing.go:69] worker 11 listening for tasks. I0129 16:51:28.245130 1 tasks_processing.go:69] worker 13 listening for tasks. I0129 16:51:28.245133 1 tasks_processing.go:69] worker 2 listening for tasks. I0129 16:51:28.245134 1 tasks_processing.go:69] worker 3 listening for tasks. I0129 16:51:28.245126 1 tasks_processing.go:69] worker 5 listening for tasks. I0129 16:51:28.245136 1 tasks_processing.go:69] worker 6 listening for tasks. I0129 16:51:28.245140 1 tasks_processing.go:69] worker 12 listening for tasks. I0129 16:51:28.245127 1 tasks_processing.go:69] worker 4 listening for tasks. I0129 16:51:28.245146 1 tasks_processing.go:69] worker 14 listening for tasks. I0129 16:51:28.245138 1 tasks_processing.go:69] worker 10 listening for tasks. I0129 16:51:28.245209 1 tasks_processing.go:71] worker 16 working on proxies task. I0129 16:51:28.245216 1 tasks_processing.go:71] worker 20 working on networks task. I0129 16:51:28.245218 1 tasks_processing.go:71] worker 27 working on config_maps task. I0129 16:51:28.245233 1 tasks_processing.go:71] worker 9 working on cluster_apiserver task. I0129 16:51:28.245239 1 tasks_processing.go:71] worker 19 working on ingress_certificates task. I0129 16:51:28.245250 1 tasks_processing.go:71] worker 10 working on version task. I0129 16:51:28.245313 1 tasks_processing.go:71] worker 2 working on validating_webhook_configurations task. I0129 16:51:28.245339 1 tasks_processing.go:71] worker 25 working on ingress task. I0129 16:51:28.245380 1 tasks_processing.go:71] worker 28 working on machine_sets task. I0129 16:51:28.245210 1 tasks_processing.go:71] worker 23 working on certificate_signing_requests task. I0129 16:51:28.245453 1 tasks_processing.go:71] worker 30 working on openshift_logging task. I0129 16:51:28.245511 1 tasks_processing.go:71] worker 18 working on operators task. I0129 16:51:28.245371 1 tasks_processing.go:71] worker 26 working on machines task. I0129 16:51:28.245668 1 tasks_processing.go:71] worker 8 working on openstack_dataplanenodesets task. I0129 16:51:28.245760 1 tasks_processing.go:71] worker 3 working on active_alerts task. I0129 16:51:28.245804 1 tasks_processing.go:71] worker 5 working on nodenetworkconfigurationpolicies task. I0129 16:51:28.245323 1 tasks_processing.go:71] worker 17 working on nodes task. I0129 16:51:28.245803 1 tasks_processing.go:71] worker 4 working on machine_autoscalers task. I0129 16:51:28.245318 1 tasks_processing.go:71] worker 29 working on openstack_version task. I0129 16:51:28.245329 1 tasks_processing.go:71] worker 24 working on container_runtime_configs task. I0129 16:51:28.245333 1 tasks_processing.go:71] worker 22 working on sap_datahubs task. I0129 16:51:28.245376 1 tasks_processing.go:71] worker 21 working on machine_configs task. I0129 16:51:28.245443 1 tasks_processing.go:71] worker 7 working on aggregated_monitoring_cr_names task. I0129 16:51:28.245450 1 tasks_processing.go:71] worker 1 working on operators_pods_and_events task. I0129 16:51:28.245792 1 tasks_processing.go:71] worker 13 working on node_logs task. I0129 16:51:28.245795 1 tasks_processing.go:71] worker 11 working on silenced_alerts task. W0129 16:51:28.246416 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0129 16:51:28.246429 1 tasks_processing.go:71] worker 11 working on pod_network_connectivity_checks task. I0129 16:51:28.245800 1 tasks_processing.go:71] worker 12 working on mutating_webhook_configurations task. I0129 16:51:28.246489 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 30.87µs to process 0 records I0129 16:51:28.245814 1 tasks_processing.go:71] worker 6 working on image_pruners task. I0129 16:51:28.245823 1 tasks_processing.go:71] worker 14 working on authentication task. W0129 16:51:28.245825 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0129 16:51:28.246724 1 tasks_processing.go:71] worker 3 working on machine_config_pools task. I0129 16:51:28.245439 1 tasks_processing.go:71] worker 0 working on openshift_machine_api_events task. I0129 16:51:28.246745 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 913.155µs to process 0 records I0129 16:51:28.253969 1 tasks_processing.go:71] worker 15 working on metrics task. I0129 16:51:28.253979 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 8.906967ms to process 0 records W0129 16:51:28.253998 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0129 16:51:28.254006 1 tasks_processing.go:71] worker 15 working on storage_classes task. I0129 16:51:28.254055 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 26.346µs to process 0 records I0129 16:51:28.257563 1 tasks_processing.go:71] worker 24 working on sap_pods task. I0129 16:51:28.257566 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 11.645318ms to process 0 records I0129 16:51:28.257686 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 12.176224ms to process 0 records E0129 16:51:28.257694 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0129 16:51:28.257700 1 gather.go:177] gatherer "clusterconfig" function "machines" took 12.04466ms to process 0 records I0129 16:51:28.257705 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 11.668771ms to process 0 records I0129 16:51:28.257710 1 tasks_processing.go:71] worker 22 working on lokistack task. I0129 16:51:28.257759 1 tasks_processing.go:71] worker 26 working on overlapping_namespace_uids task. I0129 16:51:28.257767 1 tasks_processing.go:71] worker 28 working on nodenetworkstates task. I0129 16:51:28.257822 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0129 16:51:28.257843 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0129 16:51:28.257850 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0129 16:51:28.257855 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0129 16:51:28.257872 1 controller.go:489] The operator is still being initialized I0129 16:51:28.257883 1 controller.go:512] The operator is healthy I0129 16:51:28.260870 1 tasks_processing.go:71] worker 4 working on cost_management_metrics_configs task. I0129 16:51:28.260876 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 14.990281ms to process 0 records I0129 16:51:28.260888 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 14.960553ms to process 0 records I0129 16:51:28.260895 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 15.020676ms to process 0 records E0129 16:51:28.260903 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0129 16:51:28.260909 1 tasks_processing.go:71] worker 5 working on support_secret task. I0129 16:51:28.260917 1 tasks_processing.go:71] worker 11 working on install_plans task. I0129 16:51:28.260927 1 tasks_processing.go:71] worker 29 working on ceph_cluster task. I0129 16:51:28.260911 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 14.449891ms to process 0 records I0129 16:51:28.261214 1 tasks_processing.go:71] worker 20 working on qemu_kubevirt_launcher_logs task. I0129 16:51:28.261300 1 recorder.go:75] Recording config/network with fingerprint=691da914344d3a5a4149755bb60254183ed110a2d6e60512e0f9efdfb04b7cc0 I0129 16:51:28.261321 1 gather.go:177] gatherer "clusterconfig" function "networks" took 15.699903ms to process 1 records I0129 16:51:28.261409 1 tasks_processing.go:71] worker 31 working on clusterroles task. I0129 16:51:28.261528 1 recorder.go:75] Recording config/oauth with fingerprint=d1c9bfa969a4df4cbeb67c73f6e4c2e33499fe979ff19816f346cf3819f70491 I0129 16:51:28.261541 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 16.115131ms to process 1 records I0129 16:51:28.264842 1 tasks_processing.go:71] worker 30 working on crds task. I0129 16:51:28.264850 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 19.364492ms to process 0 records I0129 16:51:28.264870 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 19.146762ms to process 0 records I0129 16:51:28.264879 1 tasks_processing.go:71] worker 8 working on pdbs task. I0129 16:51:28.265567 1 tasks_processing.go:71] worker 23 working on machine_healthchecks task. I0129 16:51:28.265579 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 20.100716ms to process 0 records I0129 16:51:28.266640 1 tasks_processing.go:71] worker 4 working on sap_config task. I0129 16:51:28.266650 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 5.757356ms to process 0 records I0129 16:51:28.266914 1 tasks_processing.go:71] worker 14 working on infrastructures task. I0129 16:51:28.267197 1 recorder.go:75] Recording config/authentication with fingerprint=88554f2d0864784ecd614fc5a15f92f2daabc5585eb0ace3f141248092cf4f21 I0129 16:51:28.267212 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 20.204479ms to process 1 records I0129 16:51:28.270054 1 tasks_processing.go:71] worker 0 working on feature_gates task. I0129 16:51:28.270059 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 23.312808ms to process 0 records I0129 16:51:28.270190 1 tasks_processing.go:71] worker 13 working on jaegers task. I0129 16:51:28.270202 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 23.866797ms to process 0 records I0129 16:51:28.270425 1 tasks_processing.go:71] worker 6 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0129 16:51:28.270725 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=47a1e3717900a3575b6ec28cac5bf2a0997a77a18033377726df00d9825643f9 I0129 16:51:28.270744 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 23.803699ms to process 1 records I0129 16:51:28.273623 1 tasks_processing.go:71] worker 29 working on olm_operators task. I0129 16:51:28.273661 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 12.684042ms to process 0 records I0129 16:51:28.273851 1 tasks_processing.go:71] worker 12 working on tsdb_status task. W0129 16:51:28.273886 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0129 16:51:28.274012 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=e2b11b087fb1e3efadb2b7b5f6a3fe3e40a76319117c0f2db54826a9451499d2 I0129 16:51:28.274052 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=70bad585e7c1bf5e3f840b53e9b014d52e46f78a194e81772db802618d44703a I0129 16:51:28.274083 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=c028b10e395771d0adfb5c1fa427404fcbb588d9b1eac8f59db06ea52101fbb2 I0129 16:51:28.274091 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 27.222106ms to process 3 records I0129 16:51:28.274102 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 26.988964ms to process 0 records I0129 16:51:28.274117 1 tasks_processing.go:71] worker 3 working on service_accounts task. I0129 16:51:28.274260 1 tasks_processing.go:71] worker 17 working on dvo_metrics task. I0129 16:51:28.274412 1 recorder.go:75] Recording config/node/ip-10-0-0-21.ec2.internal with fingerprint=c4279b4eb387780413a7cd9f08fd84cc26b4b5d1aeaeb46287b8e3524c0730e2 I0129 16:51:28.274509 1 recorder.go:75] Recording config/node/ip-10-0-1-22.ec2.internal with fingerprint=8e52a636dd596246536c4cf04b95a22b872e2b8dcbf91fd85fc644abe3b89e6c I0129 16:51:28.274591 1 recorder.go:75] Recording config/node/ip-10-0-2-244.ec2.internal with fingerprint=4137aa027f928633175631dc725520cb211660266e4d0e04fbda43c449065b54 I0129 16:51:28.274604 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 27.900144ms to process 3 records I0129 16:51:28.274679 1 recorder.go:75] Recording config/proxy with fingerprint=9b1e373dcec524fae6d5b0eaa4da7ea6920c0738f0bc7e7fe19ab64f0d363524 I0129 16:51:28.274687 1 tasks_processing.go:71] worker 16 working on image_registries task. I0129 16:51:28.274691 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 28.595707ms to process 1 records I0129 16:51:28.274698 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 32.573µs to process 0 records I0129 16:51:28.274708 1 tasks_processing.go:71] worker 12 working on container_images task. I0129 16:51:28.278430 1 tasks_processing.go:71] worker 24 working on storage_cluster task. I0129 16:51:28.278443 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 20.823287ms to process 0 records I0129 16:51:28.278464 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 11.802136ms to process 0 records I0129 16:51:28.278471 1 tasks_processing.go:71] worker 4 working on image task. I0129 16:51:28.278546 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=3cf1792edd6d147f676526b5543c060496ab13de3165e46a631a85e49ba499bc I0129 16:51:28.278564 1 tasks_processing.go:71] worker 15 working on openstack_dataplanedeployments task. I0129 16:51:28.278572 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=f4d585b62bc3d2c756de5dec45642b6b613109f4c570bad058fc24bcf44f855c I0129 16:51:28.278580 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 24.464437ms to process 2 records E0129 16:51:28.278592 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0129 16:51:28.278603 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 12.915091ms to process 0 records I0129 16:51:28.278610 1 tasks_processing.go:71] worker 23 working on monitoring_persistent_volumes task. I0129 16:51:28.281172 1 tasks_processing.go:71] worker 22 working on schedulers task. I0129 16:51:28.281183 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 23.449325ms to process 0 records I0129 16:51:28.281278 1 tasks_processing.go:74] worker 9 stopped. I0129 16:51:28.281431 1 recorder.go:75] Recording config/apiserver with fingerprint=46847f6ffe51b02cd89d3226218e02d9319000c3a63035bc5202b19bdbfa62e6 I0129 16:51:28.281443 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 36.030019ms to process 1 records I0129 16:51:28.281529 1 tasks_processing.go:74] worker 25 stopped. I0129 16:51:28.281561 1 recorder.go:75] Recording config/ingress with fingerprint=0ae91d5ce133d128a78c64c021568b59c52007f6538ba08193c955a290757b2d I0129 16:51:28.281576 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 35.893138ms to process 1 records I0129 16:51:28.283639 1 tasks_processing.go:74] worker 13 stopped. I0129 16:51:28.283646 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 13.438845ms to process 0 records I0129 16:51:28.283659 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 25.866171ms to process 0 records I0129 16:51:28.283669 1 tasks_processing.go:74] worker 28 stopped. I0129 16:51:28.284230 1 tasks_processing.go:74] worker 2 stopped. I0129 16:51:28.284245 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0129 16:51:28.284318 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0129 16:51:28.284329 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0129 16:51:28.284373 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=a66b754b19b20aadbc95023490a3b6733c2b6bb9d503e01f946cfe4f14ea8338 I0129 16:51:28.284461 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=171b861fc66e0f7589ee8555a43f322864d29c1e8d187c89688f37a524940070 I0129 16:51:28.284491 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=7d950e9d8fef631028a75ff9246560cedda51317ca089ac0e71a24abe17f80c3 I0129 16:51:28.284532 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=70d6cb58d610031827feb12376e21c9fd66cb9eed9c256316bd393ac606bcad1 I0129 16:51:28.284571 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=ab49afeecbc1ddb442d7fdbb873b95da72f5d3b932572dc150dcf9b243276703 I0129 16:51:28.284611 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=f11581bfe27960e703af32f9fcbc2324dfeb1cf429e4e05e0520ae34767b64a2 I0129 16:51:28.284659 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=eb75aa89b5eb7cf9b8ae247535a50118b2b7817b6f466476525b1047b80d16d1 I0129 16:51:28.284699 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=3a1d6d326b9a1bae186f866ca0d424eee36c4702a8acdd6f8b4a046f447b4062 I0129 16:51:28.284739 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=73f60c37dad6cae3d4d02ec1518f93cea4e4b7f766335b89a061b1d35ea01b3f I0129 16:51:28.284788 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=2fdb9142d53fbde950d7e150a8df15bc4d26da452e89f493d93dcf46c253bd99 I0129 16:51:28.284800 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 38.880133ms to process 10 records I0129 16:51:28.290831 1 tasks_processing.go:74] worker 15 stopped. I0129 16:51:28.290848 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 12.252208ms to process 0 records I0129 16:51:28.290867 1 tasks_processing.go:74] worker 5 stopped. E0129 16:51:28.290879 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0129 16:51:28.290888 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 29.947262ms to process 0 records I0129 16:51:28.290943 1 tasks_processing.go:74] worker 14 stopped. I0129 16:51:28.291559 1 recorder.go:75] Recording config/infrastructure with fingerprint=4c7905204f8551779da84db8d145320b0c4b3d1fc4dcb1e60fd054c3ed7ae6a5 I0129 16:51:28.291573 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 24.017273ms to process 1 records I0129 16:51:28.292997 1 base_controller.go:82] Caches are synced for ConfigController I0129 16:51:28.293017 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0129 16:51:28.295123 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 16.656572ms to process 0 records I0129 16:51:28.295129 1 tasks_processing.go:74] worker 24 stopped. I0129 16:51:28.309791 1 tasks_processing.go:74] worker 8 stopped. I0129 16:51:28.309900 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=eb44da0b5c2da90ec97653aed3d21b2b41ec346d98c6e0084904809f75954d29 I0129 16:51:28.309926 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=f9a1af8d46ebb11bdc5886de3f4bfaa3c32175cfebe5131646c71467167ad08b I0129 16:51:28.309974 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=29b6d4c7b3b765b9824c6b07facabf89e2d61bea94292ecbf364ff9bb67ea1b3 I0129 16:51:28.309997 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 44.90011ms to process 3 records I0129 16:51:28.310076 1 tasks_processing.go:74] worker 10 stopped. I0129 16:51:28.310340 1 recorder.go:75] Recording config/version with fingerprint=96c8fd07427036b0897eae5283bede2107b1ef5e37afcf690287a74e650e2988 I0129 16:51:28.310353 1 recorder.go:75] Recording config/id with fingerprint=678abe3ba47cf04b817a036ca33fef6cb2fe9c9b1133766ba3ee7a29e89eb14d I0129 16:51:28.310362 1 gather.go:177] gatherer "clusterconfig" function "version" took 64.737978ms to process 2 records I0129 16:51:28.310417 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=93446af751ba9edcf28519ae0a7c03d2dc666392f558bec12eccd4f8fc6898dc I0129 16:51:28.310425 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 28.998468ms to process 1 records I0129 16:51:28.310434 1 tasks_processing.go:74] worker 22 stopped. I0129 16:51:28.310545 1 gather_logs.go:145] no pods in namespace were found I0129 16:51:28.310565 1 tasks_processing.go:74] worker 20 stopped. I0129 16:51:28.310575 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 49.332814ms to process 0 records I0129 16:51:28.310682 1 tasks_processing.go:74] worker 4 stopped. I0129 16:51:28.310769 1 recorder.go:75] Recording config/image with fingerprint=cdcd33427b8db4bf2acf35712b2f99f170d0ecc133bc01735d10643fb6a19486 I0129 16:51:28.310783 1 gather.go:177] gatherer "clusterconfig" function "image" took 32.200328ms to process 1 records I0129 16:51:28.311487 1 tasks_processing.go:74] worker 23 stopped. I0129 16:51:28.311500 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 32.869915ms to process 0 records I0129 16:51:28.311515 1 tasks_processing.go:74] worker 29 stopped. I0129 16:51:28.311527 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 37.810972ms to process 0 records I0129 16:51:28.311821 1 configmapobserver.go:84] configmaps "insights-config" not found I0129 16:51:28.312005 1 tasks_processing.go:74] worker 7 stopped. I0129 16:51:28.312027 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 65.81303ms to process 0 records I0129 16:51:28.312790 1 tasks_processing.go:74] worker 26 stopped. W0129 16:51:28.312853 1 operator.go:288] started I0129 16:51:28.312864 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0129 16:51:28.312942 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 55.012157ms to process 1 records I0129 16:51:28.313295 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0129 16:51:28.313585 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0129 16:51:28.313653 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0129 16:51:28.313982 1 tasks_processing.go:74] worker 16 stopped. I0129 16:51:28.314533 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=c908a4cd462afe5690f7383f8ed70c75fa24093c4f8d2bc05ad1a725b135a60b I0129 16:51:28.314554 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 38.192888ms to process 1 records I0129 16:51:28.315116 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=b661a460824808c580fb682d0a23d38c8fb762b8a8c201cccccbd007dda5ef4f I0129 16:51:28.315310 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=11726fe2b5fd7ea399eb3efcdb12d43b282b0447a1b5fbc76faa50a220207c4b I0129 16:51:28.315322 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 52.504415ms to process 2 records I0129 16:51:28.315483 1 recorder.go:75] Recording config/featuregate with fingerprint=a08eb895541775e866b34c7b46ddba3e6553d5d5f6a83d28908ce054969d189e I0129 16:51:28.315497 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 44.53115ms to process 1 records I0129 16:51:28.315577 1 tasks_processing.go:74] worker 31 stopped. I0129 16:51:28.315836 1 tasks_processing.go:74] worker 0 stopped. I0129 16:51:28.315972 1 tasks_processing.go:74] worker 30 stopped. I0129 16:51:28.316682 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=d4879cd4644952453e61dd4c78c8541341fd60f844ae99bb96c194d5b8965b7a I0129 16:51:28.317473 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=424605bf8a2f21f2e4da49883ed53599d031418fdfe082616ade5f4ca22ec9dd I0129 16:51:28.317499 1 gather.go:177] gatherer "clusterconfig" function "crds" took 50.065042ms to process 2 records W0129 16:51:28.320716 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0129 16:51:28.327291 1 tasks_processing.go:74] worker 12 stopped. I0129 16:51:28.329699 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-t79dd with fingerprint=4bdbdb5ceeba151a9ae93fc394ce2feddff451d42b306906058cc5a8abbcd3e0 I0129 16:51:28.330356 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-gjccg with fingerprint=55dc647eb01cc28a12abdc1970baffa30c9ea5e35f77a1ef50ee898df1f3f844 I0129 16:51:28.330425 1 recorder.go:75] Recording config/running_containers with fingerprint=b9eb646771635476a5f240d2dcf88e69ce0c2996cf7caf47ea9ada12ed81ba96 I0129 16:51:28.330435 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 52.565615ms to process 3 records I0129 16:51:28.331004 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0129 16:51:28.331015 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0129 16:51:28.331020 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0129 16:51:28.331023 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0129 16:51:28.331028 1 controller.go:212] Source scaController *sca.Controller is not ready I0129 16:51:28.331044 1 controller.go:489] The operator is still being initialized I0129 16:51:28.331051 1 controller.go:512] The operator is healthy I0129 16:51:28.331351 1 tasks_processing.go:74] worker 27 stopped. E0129 16:51:28.331370 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0129 16:51:28.331382 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0129 16:51:28.331395 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0129 16:51:28.331434 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d I0129 16:51:28.331443 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0129 16:51:28.331449 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0129 16:51:28.331454 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0129 16:51:28.331516 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0129 16:51:28.331537 1 recorder.go:75] Recording config/configmaps/openshift-monitoring/cluster-monitoring-config/config with fingerprint=72b84e73f6a0b3f5cb8631772d7c6cb22f2acbacc29630102c5616778ce3c0cd I0129 16:51:28.331549 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0129 16:51:28.331557 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 86.116418ms to process 8 records I0129 16:51:28.334877 1 tasks_processing.go:74] worker 6 stopped. I0129 16:51:28.334895 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 64.438656ms to process 0 records I0129 16:51:28.335063 1 prometheus_rules.go:88] Prometheus rules successfully created E0129 16:51:28.337377 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%270ba4d43d-7626-4d24-80e0-fdda4212bfc3%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:50374->172.30.0.10:53: read: connection refused I0129 16:51:28.337390 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%270ba4d43d-7626-4d24-80e0-fdda4212bfc3%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:50374->172.30.0.10:53: read: connection refused I0129 16:51:28.382731 1 tasks_processing.go:74] worker 21 stopped. I0129 16:51:28.382754 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0129 16:51:28.382763 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 136.580751ms to process 1 records I0129 16:51:28.394751 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0129 16:51:28.398318 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:53924->172.30.0.10:53: read: connection refused I0129 16:51:28.398332 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.11:53924->172.30.0.10:53: read: connection refused I0129 16:51:28.401585 1 tasks_processing.go:74] worker 19 stopped. E0129 16:51:28.401599 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0129 16:51:28.401605 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2o4jr156ajfbgfok4qgp9fphtv1k3i6k-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2o4jr156ajfbgfok4qgp9fphtv1k3i6k-primary-cert-bundle-secret" not found I0129 16:51:28.401660 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=6e540bba9b47758c8942c7e3a588782dc05fb09e2c01e2ea4b3657408128020c I0129 16:51:28.401672 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 156.331917ms to process 1 records I0129 16:51:28.413720 1 base_controller.go:82] Caches are synced for LoggingSyncer I0129 16:51:28.413736 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0129 16:51:28.423613 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found W0129 16:51:29.316107 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0129 16:51:29.502670 1 gather_cluster_operator_pods_and_events.go:121] Found 16 pods with 18 containers I0129 16:51:29.502684 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1398101 bytes I0129 16:51:29.503568 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-xqq6r pod in namespace openshift-dns (previous: false). I0129 16:51:29.728146 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xqq6r pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-xqq6r\" is waiting to start: ContainerCreating" I0129 16:51:29.728162 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-xqq6r\" is waiting to start: ContainerCreating" I0129 16:51:29.728171 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-xqq6r pod in namespace openshift-dns (previous: false). I0129 16:51:29.738198 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0129 16:51:29.910083 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-xqq6r pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-xqq6r\" is waiting to start: ContainerCreating" I0129 16:51:29.910104 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-xqq6r\" is waiting to start: ContainerCreating" I0129 16:51:29.910115 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-4rtmd pod in namespace openshift-dns (previous: false). I0129 16:51:30.134037 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:30.134054 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-sgwkd pod in namespace openshift-dns (previous: false). W0129 16:51:30.316733 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0129 16:51:30.331129 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for node-resolver-sgwkd pod in namespace openshift-dns for failing operator dns-node-resolver (previous: false): "container \"dns-node-resolver\" in pod \"node-resolver-sgwkd\" is waiting to start: ContainerCreating" I0129 16:51:30.331144 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns-node-resolver\" in pod \"node-resolver-sgwkd\" is waiting to start: ContainerCreating" I0129 16:51:30.331154 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-tlg27 pod in namespace openshift-dns (previous: false). I0129 16:51:30.340371 1 tasks_processing.go:74] worker 18 stopped. I0129 16:51:30.340417 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=cb4cf4dd135d35ad1c921699cd5591b4deb818bb96bac2619b0a92d65dd33e0e I0129 16:51:30.340459 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=2fcfc2758ba4a702b40b8a367410153e7106423b86fb3c1c90475b350f1c3a59 I0129 16:51:30.340490 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0129 16:51:30.340514 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=bc40bbbfe031c5bdf73abf6acd25cfbf896283c93bab1a0d5c58dae8640a9bcb I0129 16:51:30.340547 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0129 16:51:30.340572 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=7686a2b325ae4503f48f1a384765cc3c383430c4e99387e96115123c903b246b I0129 16:51:30.340605 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=dd2271fe1b3e8e8d07edfba55d0e1a55e6c43e81e88258fd211f365f95b98816 I0129 16:51:30.340629 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=14bb774e39ded0ddf8f5c6f7f06dd0bfc9ae0f434bc707a4e84b61d40915647d I0129 16:51:30.340643 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=3c0abfe64f80f2fe06794753adc5d40f59915be4a4da9adb2cf71bba371b237d I0129 16:51:30.340659 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=10ce8281deeeaad01f3a7468a24606be320e89479c4e3206e55c37bfb1c86ec5 I0129 16:51:30.340668 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0129 16:51:30.340685 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=aa74782656c6d76a2ac666892b23b513d9dadfbe933eec7b95a903b958e3e58d I0129 16:51:30.340695 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0129 16:51:30.340711 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=83b29613107bd4098083649474fb472c8e7b4073cf4a53c5abe54313b5fed7f9 I0129 16:51:30.340720 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0129 16:51:30.340740 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=e76e565896ed5e4f01abbdb1cde0c2837c9ae9e00b8a12e86afcf9a6ddf8cbc3 I0129 16:51:30.340749 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0129 16:51:30.340764 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=231e6398f293bc07d6bdecddbe1e1cea5a23f1eea6eeb4ebea114a2d6225087c I0129 16:51:30.340892 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=93da9d41ff7554f998caf8eb83c65e3bb33cacaa01948ef6054bf797f2f3b492 I0129 16:51:30.340925 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0129 16:51:30.340936 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0129 16:51:30.340972 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0129 16:51:30.340995 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=3c16255b198087dd24976ee0d36b8dc2be2c505ca8dc2145962a9815d1e0310c I0129 16:51:30.341018 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=389d5063cef64b094402687d5ec113875553b808dcd1b1ad40e7e6b393f52303 I0129 16:51:30.341028 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0129 16:51:30.341044 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=c3896ac37c7a2d841d0153a960cd616639717ccd8867e8350bb142e653bce6a3 I0129 16:51:30.341053 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0129 16:51:30.341066 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=29f40e133667a03de8cf400c73cf134a6fc27a1b5ffecb76184db28b56eea96f I0129 16:51:30.341079 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=825d4acb530cdea2db86b99b50aa6b400e2ccca74da730d1d8ac6b21d5c9d684 I0129 16:51:30.341094 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=71ae73f64f208621de3a361f48d17284300a086b8e8e4bf386e7b56cddbfcceb I0129 16:51:30.341108 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=46b6d39c9749c2acd8d551126272330f9f0b1fe9f0097b06037d2867e2711ffc I0129 16:51:30.341130 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=95e2413dc33377fa9783b99dcb8fe9febc10bdb53f03132dd992b5e551d81fdd I0129 16:51:30.341139 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0129 16:51:30.341165 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=af1d23562c305afdba29b881aa43f51aa08c7bc415559c9e5cee19707766d3f3 I0129 16:51:30.341182 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0129 16:51:30.341190 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0129 16:51:30.341198 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.09480005s to process 36 records I0129 16:51:30.507645 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:30.507662 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7b6558b966-j2qjt pod in namespace openshift-image-registry (previous: false). I0129 16:51:30.703229 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:30.703248 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-dc754d5bf-h6vbj pod in namespace openshift-image-registry (previous: false). I0129 16:51:30.902998 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:30.903016 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-dc754d5bf-xrmm9 pod in namespace openshift-image-registry (previous: false). I0129 16:51:31.106111 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-dc754d5bf-xrmm9 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-dc754d5bf-xrmm9\" is waiting to start: ContainerCreating" I0129 16:51:31.106130 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-dc754d5bf-xrmm9\" is waiting to start: ContainerCreating" I0129 16:51:31.106143 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-55qnz pod in namespace openshift-image-registry (previous: false). I0129 16:51:31.309785 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for node-ca-55qnz pod in namespace openshift-image-registry for failing operator node-ca (previous: false): "container \"node-ca\" in pod \"node-ca-55qnz\" is waiting to start: ContainerCreating" I0129 16:51:31.309806 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"node-ca\" in pod \"node-ca-55qnz\" is waiting to start: ContainerCreating" I0129 16:51:31.309819 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-lwj8j pod in namespace openshift-image-registry (previous: false). W0129 16:51:31.316625 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0129 16:51:31.507704 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:31.507721 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-pbp92 pod in namespace openshift-image-registry (previous: false). I0129 16:51:31.709823 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:31.709841 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6fb849b46d-9x2rp pod in namespace openshift-ingress (previous: false). I0129 16:51:31.911839 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-6fb849b46d-9x2rp pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-6fb849b46d-9x2rp\" is waiting to start: ContainerCreating" I0129 16:51:31.911862 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-6fb849b46d-9x2rp\" is waiting to start: ContainerCreating" I0129 16:51:31.911876 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-6fb849b46d-xmqmr pod in namespace openshift-ingress (previous: false). I0129 16:51:32.102939 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0129 16:51:32.102979 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-8699f986df-76bb6 pod in namespace openshift-ingress (previous: false). I0129 16:51:32.306641 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-8699f986df-76bb6 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-8699f986df-76bb6\" is waiting to start: ContainerCreating" I0129 16:51:32.306661 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-8699f986df-76bb6\" is waiting to start: ContainerCreating" I0129 16:51:32.306684 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-dg2pf pod in namespace openshift-ingress-canary (previous: false). W0129 16:51:32.317242 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0129 16:51:32.534861 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-dg2pf pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dg2pf\" is waiting to start: ContainerCreating" I0129 16:51:32.534882 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-dg2pf\" is waiting to start: ContainerCreating" I0129 16:51:32.534894 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-jjfsq pod in namespace openshift-kube-storage-version-migrator (previous: false). I0129 16:51:32.710278 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-jjfsq pod in namespace openshift-kube-storage-version-migrator (previous: false). I0129 16:51:32.909138 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-khz6w pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0129 16:51:33.114007 1 tasks_processing.go:74] worker 1 stopped. I0129 16:51:33.114129 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=e1d031a16d0937946e9600afb6da5606f2d8cdeb9fd7e30312efff5281ea3369 I0129 16:51:33.114166 1 recorder.go:75] Recording events/openshift-dns with fingerprint=3926477dfb8379aeee022690e87b8b1ca6be82a241ccb860c180172027e580d8 I0129 16:51:33.114239 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=7bd150c6d478cff7b59cd66102285f9a38789fbd6e0d2bb20c6c819a96fe6d2e I0129 16:51:33.114273 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=0c2b02ba82e3da816a72fabe6b3a7fb79c00df526e35fd80b89a8bd9add148cb I0129 16:51:33.114315 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=8581ebf0bf8ed3794f6f4c9247a4153726ace0952eb33d7389e8c7bc4bccf2cd I0129 16:51:33.114325 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=089887c7da91c9c1dd9098ace022240c0ea2149129097a0af23c375f664c0a5e I0129 16:51:33.114336 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=f29d4adedb496247c892007a09bdead09f56f20e680dcb4939be05963b2b566f I0129 16:51:33.114383 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=32dfb3514d1dd5e6da0ec8f80c3b7ca6f4a2494714ae2884e3521f62297076bf I0129 16:51:33.114397 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-jjfsq/migrator_current.log with fingerprint=5f59e5df6b218aad6b777abe37bea4e06836dbfa4f51bbc68942a86ece8795e5 I0129 16:51:33.114403 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-jjfsq/graceful-termination_current.log with fingerprint=cc189fb228490c88dd8f064e0679c132b46a5bb4d5523bd7bfa7de6cf53beacf I0129 16:51:33.114475 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-khz6w/kube-storage-version-migrator-operator_current.log with fingerprint=7efa6e303e1d07caa701457e2fe661ec5d54c2d923ff9602e7785cd1e8c8e2ca I0129 16:51:33.114484 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.867693614s to process 11 records W0129 16:51:33.315914 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0129 16:51:33.315937 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0129 16:51:33.315968 1 tasks_processing.go:74] worker 17 stopped. E0129 16:51:33.315978 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0129 16:51:33.315990 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0129 16:51:33.316005 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0129 16:51:33.316016 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.041689826s to process 1 records I0129 16:51:40.701043 1 configmapobserver.go:84] configmaps "insights-config" not found I0129 16:51:40.718643 1 tasks_processing.go:74] worker 11 stopped. I0129 16:51:40.718677 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0129 16:51:40.718690 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.457710678s to process 1 records I0129 16:51:40.899255 1 configmapobserver.go:84] configmaps "insights-config" not found I0129 16:51:41.104596 1 configmapobserver.go:84] configmaps "insights-config" not found I0129 16:51:41.479515 1 tasks_processing.go:74] worker 3 stopped. I0129 16:51:41.479782 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=eb1106d6d263e004815aa679a0b6f0f4793e0770d2f0099ce342124ca07f4cc4 I0129 16:51:41.479798 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.205376057s to process 1 records E0129 16:51:41.479853 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.234s with: function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0129 16:51:41.480959 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0129 16:51:41.480974 1 periodic.go:209] Running workloads gatherer I0129 16:51:41.480988 1 tasks_processing.go:45] number of workers: 2 I0129 16:51:41.481001 1 tasks_processing.go:69] worker 1 listening for tasks. I0129 16:51:41.481005 1 tasks_processing.go:71] worker 1 working on workload_info task. I0129 16:51:41.481016 1 tasks_processing.go:69] worker 0 listening for tasks. I0129 16:51:41.481027 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0129 16:51:41.507556 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 21s for image data I0129 16:51:41.509696 1 tasks_processing.go:74] worker 0 stopped. I0129 16:51:41.509712 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 28.65723ms to process 0 records I0129 16:51:41.516462 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0129 16:51:41.527340 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (11ms) I0129 16:51:41.536565 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (9ms) I0129 16:51:41.546605 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (10ms) I0129 16:51:41.555189 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (9ms) I0129 16:51:41.563794 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (9ms) I0129 16:51:41.572561 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (9ms) I0129 16:51:41.580999 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (8ms) I0129 16:51:41.589459 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (8ms) I0129 16:51:41.600068 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (11ms) I0129 16:51:41.621391 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (21ms) I0129 16:51:41.716779 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (95ms) I0129 16:51:41.816619 1 gather_workloads_info.go:387] No image sha256:f82357030795138d2081ecc5172092222b0f4faea27e9a7a0474fbeae29111ad (100ms) I0129 16:51:41.920631 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (104ms) I0129 16:51:42.019929 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0129 16:51:42.117879 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (98ms) I0129 16:51:42.216443 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (99ms) I0129 16:51:42.316487 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0129 16:51:42.417202 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (101ms) I0129 16:51:42.517123 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0129 16:51:42.617125 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (100ms) I0129 16:51:42.716301 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (99ms) I0129 16:51:42.816514 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0129 16:51:42.917104 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (101ms) I0129 16:51:43.017367 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0129 16:51:43.116852 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (99ms) I0129 16:51:43.216862 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0129 16:51:43.317272 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0129 16:51:43.416566 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (99ms) I0129 16:51:43.516547 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0129 16:51:43.616362 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (100ms) I0129 16:51:43.718743 1 gather_workloads_info.go:387] No image sha256:ef2bfb546cd8aac31b7671585f718f44b268ed43a9c8562016ab741558a7dfa1 (102ms) I0129 16:51:43.816966 1 gather_workloads_info.go:387] No image sha256:fac526ab9335a9a8decb028a30673f4ae718deae8da114f36103b6438ba6458f (98ms) I0129 16:51:43.917412 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (100ms) I0129 16:51:43.917457 1 tasks_processing.go:74] worker 1 stopped. E0129 16:51:43.917474 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0129 16:51:43.917871 1 recorder.go:75] Recording config/workload_info with fingerprint=a200ae45536b36676645f157f0b7990447ead9d12a22d944adc35207eaddaae1 I0129 16:51:43.917891 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.436442071s to process 1 records E0129 16:51:43.917915 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.436s with: function \"workload_info\" failed with an error" I0129 16:51:43.919041 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0129 16:51:43.919056 1 periodic.go:209] Running conditional gatherer I0129 16:51:43.924468 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0129 16:51:43.933477 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.11:38300->172.30.0.10:53: read: connection refused E0129 16:51:43.933832 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0129 16:51:43.933908 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0129 16:51:43.947052 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0129 16:51:43.947067 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947075 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947081 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947085 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947089 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947094 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947098 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947103 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0129 16:51:43.947107 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0129 16:51:43.947131 1 tasks_processing.go:45] number of workers: 3 I0129 16:51:43.947147 1 tasks_processing.go:69] worker 2 listening for tasks. I0129 16:51:43.947154 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0129 16:51:43.947155 1 tasks_processing.go:69] worker 0 listening for tasks. I0129 16:51:43.947165 1 tasks_processing.go:69] worker 1 listening for tasks. I0129 16:51:43.947175 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0129 16:51:43.947180 1 tasks_processing.go:74] worker 0 stopped. I0129 16:51:43.947180 1 tasks_processing.go:71] worker 1 working on conditional_gatherer_rules task. I0129 16:51:43.947242 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0129 16:51:43.947255 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.091µs to process 1 records I0129 16:51:43.947322 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0129 16:51:43.947335 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.08µs to process 1 records I0129 16:51:43.947341 1 tasks_processing.go:74] worker 1 stopped. I0129 16:51:43.947438 1 tasks_processing.go:74] worker 2 stopped. I0129 16:51:43.947454 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 277.998µs to process 0 records I0129 16:51:43.947487 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.11:38300->172.30.0.10:53: read: connection refused I0129 16:51:43.947513 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0129 16:51:43.969862 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=460bea702692cdf050b387c911829486c4e47a1b95a88cf80c4f377ae79e6caf I0129 16:51:43.969986 1 diskrecorder.go:70] Writing 107 records to /var/lib/insights-operator/insights-2026-01-29-165143.tar.gz I0129 16:51:43.976857 1 diskrecorder.go:51] Wrote 107 records to disk in 6ms I0129 16:51:43.976888 1 periodic.go:278] Gathering cluster info every 2h0m0s I0129 16:51:43.976910 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0129 16:51:56.647000 1 configmapobserver.go:84] configmaps "insights-config" not found I0129 16:52:42.340602 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="a3919006ba945554300451ab193a44905e0a4ea02112dcaf0d6d5b0bea914b62") W0129 16:52:42.340633 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0129 16:52:42.340694 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="07008d9aee89595debc188f906ef1edb0d7ce73212235c136fc98d6f8de0a7da") I0129 16:52:42.340697 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0129 16:52:42.340723 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0129 16:52:42.340751 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0129 16:52:42.340781 1 requestheader_controller.go:194] Shutting down RequestHeaderAuthRequestController I0129 16:52:42.340776 1 base_controller.go:181] Shutting down LoggingSyncer ... I0129 16:52:42.340795 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="687a039700e19e01c04f3da70e833de7573a5a7b4f6cab61ee891618389aa479") I0129 16:52:42.340793 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" I0129 16:52:42.340804 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0129 16:52:42.340811 1 periodic.go:170] Shutting down I0129 16:52:42.340815 1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ... I0129 16:52:42.340821 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0129 16:52:42.340826 1 secure_serving.go:258] Stopped listening on [::]:8443 I0129 16:52:42.340828 1 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0129 16:52:42.340848 1 genericapiserver.go:600] "[graceful-termination] shutdown event" name="HTTPServerStoppedListening" I0129 16:52:42.340851 1 dynamic_serving_content.go:149] "Shutting down controller" name="serving-cert::/tmp/serving-cert-1927861705/tls.crt::/tmp/serving-cert-1927861705/tls.key" I0129 16:52:42.340873 1 tlsconfig.go:258] "Shutting down DynamicServingCertificateController" I0129 16:52:42.340887 1 genericapiserver.go:713] [graceful-termination] apiserver is exiting