W0705 02:13:35.727408 1 cmd.go:257] Using insecure, self-signed certificates I0705 02:13:36.273112 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 02:13:36.273375 1 observer_polling.go:159] Starting file observer I0705 02:13:37.524508 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0705 02:13:37.524704 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0705 02:13:37.525233 1 secure_serving.go:57] Forcing use of http/1.1 only W0705 02:13:37.525251 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0705 02:13:37.525255 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0705 02:13:37.525259 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0705 02:13:37.525261 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0705 02:13:37.525264 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0705 02:13:37.525266 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0705 02:13:37.525336 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0705 02:13:37.528807 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0705 02:13:37.528833 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"3a362a46-6a9f-4d1d-9d2d-4abdcf1c59e3", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0705 02:13:37.528972 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0705 02:13:37.528986 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0705 02:13:37.528992 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0705 02:13:37.529000 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0705 02:13:37.529035 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0705 02:13:37.529041 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0705 02:13:37.529346 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-883320081/tls.crt::/tmp/serving-cert-883320081/tls.key" I0705 02:13:37.529636 1 secure_serving.go:213] Serving securely on [::]:8443 I0705 02:13:37.529662 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0705 02:13:37.542131 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0705 02:13:37.542158 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0705 02:13:37.542265 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0705 02:13:37.552355 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:13:37.552372 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0705 02:13:37.562125 1 secretconfigobserver.go:119] support secret does not exist I0705 02:13:37.568796 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0705 02:13:37.575184 1 secretconfigobserver.go:119] support secret does not exist I0705 02:13:37.575339 1 recorder.go:161] Pruning old reports every 6h29m28s, max age is 288h0m0s I0705 02:13:37.579892 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0705 02:13:37.579901 1 periodic.go:209] Running clusterconfig gatherer I0705 02:13:37.579915 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0705 02:13:37.579947 1 tasks_processing.go:45] number of workers: 32 I0705 02:13:37.579956 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0705 02:13:37.579968 1 insightsreport.go:296] Starting report retriever I0705 02:13:37.579972 1 tasks_processing.go:69] worker 2 listening for tasks. I0705 02:13:37.579974 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 02:13:37.579977 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0705 02:13:37.579984 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 02:13:37.579987 1 tasks_processing.go:71] worker 0 working on ingress task. I0705 02:13:37.579989 1 tasks_processing.go:71] worker 1 working on dvo_metrics task. I0705 02:13:37.579993 1 tasks_processing.go:69] worker 11 listening for tasks. I0705 02:13:37.579989 1 tasks_processing.go:69] worker 6 listening for tasks. I0705 02:13:37.579993 1 tasks_processing.go:69] worker 8 listening for tasks. I0705 02:13:37.580017 1 tasks_processing.go:69] worker 4 listening for tasks. I0705 02:13:37.580025 1 tasks_processing.go:69] worker 7 listening for tasks. I0705 02:13:37.580029 1 tasks_processing.go:69] worker 3 listening for tasks. I0705 02:13:37.580030 1 tasks_processing.go:69] worker 10 listening for tasks. I0705 02:13:37.580021 1 tasks_processing.go:69] worker 9 listening for tasks. I0705 02:13:37.580035 1 tasks_processing.go:69] worker 5 listening for tasks. I0705 02:13:37.580036 1 tasks_processing.go:69] worker 22 listening for tasks. I0705 02:13:37.580041 1 tasks_processing.go:69] worker 12 listening for tasks. I0705 02:13:37.580048 1 tasks_processing.go:69] worker 18 listening for tasks. I0705 02:13:37.580045 1 tasks_processing.go:69] worker 27 listening for tasks. I0705 02:13:37.580046 1 tasks_processing.go:69] worker 17 listening for tasks. I0705 02:13:37.580054 1 tasks_processing.go:69] worker 19 listening for tasks. I0705 02:13:37.580052 1 tasks_processing.go:69] worker 23 listening for tasks. I0705 02:13:37.580055 1 tasks_processing.go:69] worker 13 listening for tasks. I0705 02:13:37.580059 1 tasks_processing.go:69] worker 20 listening for tasks. I0705 02:13:37.580060 1 tasks_processing.go:69] worker 24 listening for tasks. I0705 02:13:37.580064 1 tasks_processing.go:69] worker 14 listening for tasks. I0705 02:13:37.580067 1 tasks_processing.go:69] worker 21 listening for tasks. I0705 02:13:37.580068 1 tasks_processing.go:69] worker 30 listening for tasks. I0705 02:13:37.580068 1 tasks_processing.go:69] worker 25 listening for tasks. I0705 02:13:37.580073 1 tasks_processing.go:69] worker 16 listening for tasks. I0705 02:13:37.580073 1 tasks_processing.go:69] worker 15 listening for tasks. I0705 02:13:37.580075 1 tasks_processing.go:69] worker 26 listening for tasks. I0705 02:13:37.580080 1 tasks_processing.go:69] worker 28 listening for tasks. I0705 02:13:37.580081 1 tasks_processing.go:69] worker 31 listening for tasks. I0705 02:13:37.580083 1 tasks_processing.go:71] worker 2 working on operators_pods_and_events task. I0705 02:13:37.580087 1 tasks_processing.go:69] worker 29 listening for tasks. I0705 02:13:37.580155 1 tasks_processing.go:71] worker 29 working on monitoring_persistent_volumes task. I0705 02:13:37.580162 1 tasks_processing.go:71] worker 9 working on machine_configs task. I0705 02:13:37.580165 1 tasks_processing.go:71] worker 11 working on openshift_logging task. I0705 02:13:37.580229 1 tasks_processing.go:71] worker 4 working on silenced_alerts task. I0705 02:13:37.580276 1 tasks_processing.go:71] worker 3 working on ingress_certificates task. W0705 02:13:37.580298 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 02:13:37.580309 1 tasks_processing.go:71] worker 4 working on install_plans task. I0705 02:13:37.580419 1 tasks_processing.go:71] worker 18 working on openstack_controlplanes task. I0705 02:13:37.580498 1 tasks_processing.go:71] worker 21 working on container_runtime_configs task. I0705 02:13:37.580539 1 tasks_processing.go:71] worker 14 working on openstack_version task. I0705 02:13:37.580579 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 27.979µs to process 0 records I0705 02:13:37.580160 1 tasks_processing.go:71] worker 10 working on config_maps task. I0705 02:13:37.580494 1 tasks_processing.go:71] worker 6 working on storage_classes task. I0705 02:13:37.580544 1 tasks_processing.go:71] worker 15 working on machine_config_pools task. I0705 02:13:37.580242 1 tasks_processing.go:71] worker 8 working on storage_cluster task. I0705 02:13:37.580257 1 tasks_processing.go:71] worker 7 working on crds task. I0705 02:13:37.580263 1 tasks_processing.go:71] worker 12 working on machine_autoscalers task. I0705 02:13:37.580267 1 tasks_processing.go:71] worker 5 working on machine_healthchecks task. I0705 02:13:37.580271 1 tasks_processing.go:71] worker 22 working on schedulers task. I0705 02:13:37.580509 1 tasks_processing.go:71] worker 13 working on pdbs task. I0705 02:13:37.580514 1 tasks_processing.go:71] worker 19 working on support_secret task. I0705 02:13:37.580518 1 tasks_processing.go:71] worker 23 working on authentication task. I0705 02:13:37.580524 1 tasks_processing.go:71] worker 27 working on nodenetworkstates task. I0705 02:13:37.580529 1 tasks_processing.go:71] worker 24 working on validating_webhook_configurations task. I0705 02:13:37.580533 1 tasks_processing.go:71] worker 20 working on certificate_signing_requests task. I0705 02:13:37.580549 1 tasks_processing.go:71] worker 30 working on lokistack task. I0705 02:13:37.580554 1 tasks_processing.go:71] worker 25 working on image task. I0705 02:13:37.580558 1 tasks_processing.go:71] worker 16 working on operators task. I0705 02:13:37.580565 1 tasks_processing.go:71] worker 28 working on oauths task. I0705 02:13:37.580568 1 tasks_processing.go:71] worker 31 working on openshift_machine_api_events task. I0705 02:13:37.580158 1 tasks_processing.go:71] worker 17 working on machines task. I0705 02:13:37.580568 1 tasks_processing.go:71] worker 26 working on service_accounts task. I0705 02:13:37.593897 1 tasks_processing.go:71] worker 8 working on sap_datahubs task. I0705 02:13:37.593907 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 13.002741ms to process 0 records I0705 02:13:37.593925 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 13.355944ms to process 0 records I0705 02:13:37.593932 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 13.404135ms to process 0 records I0705 02:13:37.593937 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 13.744153ms to process 0 records I0705 02:13:37.593946 1 tasks_processing.go:71] worker 11 working on cost_management_metrics_configs task. I0705 02:13:37.594060 1 tasks_processing.go:71] worker 18 working on node_logs task. I0705 02:13:37.594091 1 tasks_processing.go:71] worker 21 working on openstack_dataplanenodesets task. I0705 02:13:37.598392 1 tasks_processing.go:71] worker 27 working on active_alerts task. W0705 02:13:37.598420 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 02:13:37.598445 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 16.932597ms to process 0 records I0705 02:13:37.598455 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 25.607µs to process 0 records I0705 02:13:37.598461 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 17.861491ms to process 0 records I0705 02:13:37.598468 1 tasks_processing.go:71] worker 14 working on feature_gates task. E0705 02:13:37.598521 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0705 02:13:37.598539 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 17.241256ms to process 0 records I0705 02:13:37.598584 1 tasks_processing.go:71] worker 19 working on ceph_cluster task. I0705 02:13:37.598707 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=435f08265a12f6cdc4e516ce5ce5f6ff890db3dcf86ba8f089531cd7e9eb1631 I0705 02:13:37.598611 1 tasks_processing.go:71] worker 27 working on jaegers task. I0705 02:13:37.598803 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0705 02:13:37.598862 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0705 02:13:37.598894 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0705 02:13:37.598922 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0705 02:13:37.598716 1 tasks_processing.go:71] worker 6 working on container_images task. I0705 02:13:37.598772 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=7dfaadd1e287a0d1dc7371074a8ef1ec81beef9e7a18dd8851ace3717d4c457b I0705 02:13:37.599344 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 17.756951ms to process 2 records I0705 02:13:37.599384 1 controller.go:489] The operator is still being initialized I0705 02:13:37.599439 1 controller.go:512] The operator is healthy I0705 02:13:37.599388 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 17.515767ms to process 0 records E0705 02:13:37.600163 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0705 02:13:37.600196 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 17.646153ms to process 0 records E0705 02:13:37.600225 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0705 02:13:37.600258 1 gather.go:177] gatherer "clusterconfig" function "machines" took 16.677805ms to process 0 records I0705 02:13:37.600292 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 17.172426ms to process 0 records I0705 02:13:37.600331 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 17.994203ms to process 0 records I0705 02:13:37.600360 1 tasks_processing.go:71] worker 17 working on sap_config task. I0705 02:13:37.600268 1 tasks_processing.go:71] worker 5 working on proxies task. I0705 02:13:37.599405 1 tasks_processing.go:71] worker 12 working on mutating_webhook_configurations task. I0705 02:13:37.600927 1 recorder.go:75] Recording config/authentication with fingerprint=02243a307d46f8c0ca69b467f3ca8afa4037bee2bca33cbdbdc7daa01163921c I0705 02:13:37.600947 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 17.534181ms to process 1 records I0705 02:13:37.601048 1 tasks_processing.go:71] worker 15 working on version task. I0705 02:13:37.601103 1 tasks_processing.go:71] worker 30 working on olm_operators task. I0705 02:13:37.601288 1 recorder.go:75] Recording config/ingress with fingerprint=a71c0ce6e4db5ebdf3b5e9ab9349ca4105f60cb63959d965ad3968618922bb58 I0705 02:13:37.601296 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 19.249265ms to process 1 records I0705 02:13:37.601378 1 tasks_processing.go:71] worker 23 working on image_registries task. I0705 02:13:37.601420 1 tasks_processing.go:71] worker 0 working on nodes task. I0705 02:13:37.601420 1 recorder.go:75] Recording config/image with fingerprint=8fefa573b9d127e1a594725d903a9851fba6b007a705bd56a397f3d25e2e000b I0705 02:13:37.601463 1 tasks_processing.go:71] worker 25 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0705 02:13:37.601477 1 gather.go:177] gatherer "clusterconfig" function "image" took 19.039286ms to process 1 records I0705 02:13:37.601537 1 tasks_processing.go:71] worker 28 working on overlapping_namespace_uids task. I0705 02:13:37.602426 1 recorder.go:75] Recording config/oauth with fingerprint=e4a2247c920f2b45fa30eb290e94cc50d0b0670e9a1d30a41e51b1deaf0e0ac8 I0705 02:13:37.602479 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 19.207546ms to process 1 records I0705 02:13:37.602538 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 21.461746ms to process 0 records I0705 02:13:37.602574 1 tasks_processing.go:71] worker 29 working on sap_pods task. I0705 02:13:37.602705 1 tasks_processing.go:71] worker 13 working on image_pruners task. I0705 02:13:37.602761 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=b18c22a99988742dde89e7345c16bc48fb4e7f2dd26fdfbcd3f1af7551b66b09 I0705 02:13:37.602840 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=10024b375ba5ffa8870214cc69ff43da7fc54ec5de89f5f3eef64acb11ea036f I0705 02:13:37.602870 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=cb17a32a26c88353bc081cde90fd78a54dbec79d83696f0a788b90d5e9a08a2f I0705 02:13:37.602893 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 20.545277ms to process 3 records I0705 02:13:37.602902 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 8.506957ms to process 0 records I0705 02:13:37.602916 1 tasks_processing.go:71] worker 8 working on aggregated_monitoring_cr_names task. I0705 02:13:37.602971 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=54a4b3f845e0e62028851cbe6d90f4ed983db1511246335d9c31f22f740a26e7 I0705 02:13:37.602985 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 21.246294ms to process 1 records I0705 02:13:37.602992 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 20.409257ms to process 0 records I0705 02:13:37.602997 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 8.356698ms to process 0 records I0705 02:13:37.603005 1 tasks_processing.go:71] worker 21 working on cluster_apiserver task. I0705 02:13:37.603041 1 tasks_processing.go:71] worker 31 working on infrastructures task. I0705 02:13:37.603072 1 tasks_processing.go:71] worker 22 working on networks task. I0705 02:13:37.603240 1 tasks_processing.go:71] worker 11 working on pod_network_connectivity_checks task. I0705 02:13:37.603329 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 9.28676ms to process 0 records I0705 02:13:37.604252 1 tasks_processing.go:71] worker 24 working on qemu_kubevirt_launcher_logs task. I0705 02:13:37.604753 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=93601d623dea92f6daaac206e6d37ee5ab89c0b6fb158858ffad0aefd5ed5473 I0705 02:13:37.604879 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=b47d920ba621c79b4a52e3094c0ecb8ebb975feeb53a6b6670f4b738e83a8c22 I0705 02:13:37.604916 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=0db3c79ad0e98f577c4b2577c37cc80e4fff0efadfffbf18dda59f6e8d21bd04 I0705 02:13:37.604965 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=7ca840c12e6e70f3b42108958383d3b28c317965d5dba0de8f726e1e4f052249 I0705 02:13:37.605012 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=a1454d4f7f852ec72ae2a700cde5eb993b1a9d56576eec3eab59f4f428763e03 I0705 02:13:37.605247 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=8c6e45f95efcc98073ad7e25ad13bd1c09a94b0d83b226eb3bfcc948ab02a502 I0705 02:13:37.605363 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=b69d2d87cb5515f61ab66fefdb5f091866206a4987ac2801993206f6a06e6f6a I0705 02:13:37.605485 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=d1f20dbd948f9e1f6c6b0cedfedd268ad1ab02a5c8a9bf6e15b8c347725c713a I0705 02:13:37.605747 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=0661bc99e5132df164305cf09f1a415b8491dd11360e79c0877de3ff686f9acf I0705 02:13:37.605854 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=e165a0625de8fa04ab2539697588b816a4686879371a1c7bece646f960afba3c I0705 02:13:37.605953 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=a19cd2c78019853f7377d15cf2a84d744deac0f9864ad98cd177a9e1a311fed9 I0705 02:13:37.605984 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 22.724226ms to process 11 records I0705 02:13:37.606014 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 23.86792ms to process 0 records I0705 02:13:37.606043 1 tasks_processing.go:71] worker 20 working on nodenetworkconfigurationpolicies task. I0705 02:13:37.607385 1 tasks_processing.go:71] worker 19 working on metrics task. W0705 02:13:37.607424 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 02:13:37.607425 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 8.760121ms to process 0 records I0705 02:13:37.607448 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 13.344413ms to process 0 records I0705 02:13:37.607460 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 8.618909ms to process 0 records I0705 02:13:37.607468 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 34.67µs to process 0 records I0705 02:13:37.607479 1 tasks_processing.go:71] worker 19 working on clusterroles task. I0705 02:13:37.607727 1 tasks_processing.go:71] worker 27 working on machine_sets task. I0705 02:13:37.607785 1 tasks_processing.go:71] worker 18 working on openstack_dataplanedeployments task. I0705 02:13:37.607916 1 tasks_processing.go:71] worker 14 working on tsdb_status task. W0705 02:13:37.607983 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 02:13:37.608205 1 recorder.go:75] Recording config/featuregate with fingerprint=4f801961c9ff97a197bfbfdf45e7231aba205bf176116c19e63f730b6f2eb686 I0705 02:13:37.608227 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 9.43336ms to process 1 records I0705 02:13:37.608250 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 58.428µs to process 0 records I0705 02:13:37.608262 1 tasks_processing.go:74] worker 14 stopped. W0705 02:13:37.609928 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 02:13:37.616332 1 tasks_processing.go:74] worker 17 stopped. I0705 02:13:37.616347 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 15.951008ms to process 0 records I0705 02:13:37.616728 1 tasks_processing.go:74] worker 5 stopped. I0705 02:13:37.616834 1 recorder.go:75] Recording config/proxy with fingerprint=de3d5a32ee014a2b11a3f71835832d55cb2c273e59b494a11d8ee28d590aebaa I0705 02:13:37.616850 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 16.173026ms to process 1 records I0705 02:13:37.626497 1 tasks_processing.go:74] worker 29 stopped. I0705 02:13:37.626513 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 23.900666ms to process 0 records I0705 02:13:37.626635 1 tasks_processing.go:74] worker 30 stopped. I0705 02:13:37.626649 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 25.519623ms to process 0 records I0705 02:13:37.626925 1 tasks_processing.go:74] worker 0 stopped. I0705 02:13:37.627250 1 recorder.go:75] Recording config/node/ip-10-0-0-209.ec2.internal with fingerprint=dc13765502208eda65f4c05521e0bed185c07408529df05175037ca78722af20 I0705 02:13:37.627342 1 recorder.go:75] Recording config/node/ip-10-0-1-100.ec2.internal with fingerprint=471afbbe693d23bc6be780cda14d8fdb562dbce8b526720339dbdbfb401d6630 I0705 02:13:37.627463 1 recorder.go:75] Recording config/node/ip-10-0-2-161.ec2.internal with fingerprint=a92e73c2aaab0070c0f0822554de4dec15fc025620b81848b0f6e0c8da09a6e4 I0705 02:13:37.627473 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 25.489224ms to process 3 records I0705 02:13:37.627563 1 tasks_processing.go:74] worker 23 stopped. I0705 02:13:37.628051 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=d82ab31d9f09f41b664b72bb7b11d749696e6696bc8c06912014bb800674d1b6 I0705 02:13:37.628075 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 25.711078ms to process 1 records I0705 02:13:37.629650 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0705 02:13:37.629657 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0705 02:13:37.629725 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0705 02:13:37.631299 1 tasks_processing.go:74] worker 6 stopped. I0705 02:13:37.634151 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-wrwzq with fingerprint=c1aa439cd9b0680619e3792083f735944cd62b881976a446de2adfe151fb6ea4 I0705 02:13:37.634278 1 recorder.go:75] Recording config/running_containers with fingerprint=20e88edd1eca10434ee130bd3513a34109cb7daab89d0494323937df461b7c6a I0705 02:13:37.634318 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 32.100616ms to process 2 records I0705 02:13:37.637480 1 tasks_processing.go:74] worker 11 stopped. E0705 02:13:37.637491 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0705 02:13:37.637500 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 34.227133ms to process 0 records I0705 02:13:37.637513 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 31.430462ms to process 0 records I0705 02:13:37.637522 1 tasks_processing.go:74] worker 20 stopped. I0705 02:13:37.637795 1 tasks_processing.go:74] worker 18 stopped. I0705 02:13:37.637889 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 29.93853ms to process 0 records I0705 02:13:37.637906 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 30.08513ms to process 0 records I0705 02:13:37.637914 1 tasks_processing.go:74] worker 27 stopped. I0705 02:13:37.638010 1 tasks_processing.go:74] worker 22 stopped. I0705 02:13:37.638148 1 recorder.go:75] Recording config/network with fingerprint=d13f1c15ba1ce95c5448e7068d6d8515d00307557a68bf97e73078e357917af5 I0705 02:13:37.638165 1 gather.go:177] gatherer "clusterconfig" function "networks" took 34.887952ms to process 1 records I0705 02:13:37.638250 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=976eb465c2b8c66355658fe2228f77d4d5d6d8cb47388521f279a4f3d2ebc6d1 I0705 02:13:37.638287 1 tasks_processing.go:74] worker 13 stopped. I0705 02:13:37.638292 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 35.333889ms to process 1 records I0705 02:13:37.638315 1 tasks_processing.go:74] worker 31 stopped. I0705 02:13:37.638789 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0705 02:13:37.638864 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s W0705 02:13:37.638921 1 operator.go:288] started I0705 02:13:37.638982 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0705 02:13:37.639099 1 recorder.go:75] Recording config/infrastructure with fingerprint=6ead2eca5d97e053666efded034a9a482ccb06d03b7cb6a40aeee45a12d9300c I0705 02:13:37.639116 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 35.241629ms to process 1 records I0705 02:13:37.639178 1 tasks_processing.go:74] worker 7 stopped. I0705 02:13:37.639759 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=1d2c9721b9162610ff7fc9682f4b4b9a1bf03298b6ea07eaa6496e9798ee343f I0705 02:13:37.639974 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=3a70bcd314a0a3219ee8ec02a052499c7a5a307a41dd680a8d138eecd97faa1f I0705 02:13:37.639984 1 gather.go:177] gatherer "clusterconfig" function "crds" took 57.481773ms to process 2 records I0705 02:13:37.640003 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0705 02:13:37.640010 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 58.334491ms to process 1 records I0705 02:13:37.640074 1 tasks_processing.go:74] worker 9 stopped. I0705 02:13:37.640091 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=9843c435934d885d45d70753dfb5c5a8f0097522bb0d0fbffc959f45d267d94f I0705 02:13:37.640120 1 tasks_processing.go:74] worker 12 stopped. I0705 02:13:37.640129 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=86d757294c5a97cd60fd1511bfc9f4fad44d19594c01f1b53bde020c06923b3a I0705 02:13:37.640159 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=7c48aaacf029d7d41e29d1665264cfccc725aff44d13979ba275f643defa4b38 I0705 02:13:37.640168 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 37.966243ms to process 3 records I0705 02:13:37.640247 1 tasks_processing.go:74] worker 21 stopped. I0705 02:13:37.640279 1 recorder.go:75] Recording config/apiserver with fingerprint=745834fc5284a1a389169e334ffd140e8b9f2d7872fd5c30bc1b4502c7e46334 I0705 02:13:37.640288 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 35.856145ms to process 1 records I0705 02:13:37.642313 1 base_controller.go:82] Caches are synced for ConfigController I0705 02:13:37.642323 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0705 02:13:37.646201 1 tasks_processing.go:74] worker 19 stopped. I0705 02:13:37.646369 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=b6327ade1c139e0e34128fb0f8205f5fc9e0b1464826633e7c49064134d2ca66 I0705 02:13:37.646452 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=1b695fd410934a1f0aaf28ebba54bef7224218d462a8595d2acf54cfe74f6f79 I0705 02:13:37.646460 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 38.710519ms to process 2 records I0705 02:13:37.650240 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0705 02:13:37.650252 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0705 02:13:37.650256 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0705 02:13:37.650261 1 controller.go:212] Source scaController *sca.Controller is not ready I0705 02:13:37.650266 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0705 02:13:37.650286 1 controller.go:489] The operator is still being initialized I0705 02:13:37.650294 1 controller.go:512] The operator is healthy I0705 02:13:37.653802 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:13:37.654000 1 prometheus_rules.go:88] Prometheus rules successfully created I0705 02:13:37.656869 1 tasks_processing.go:74] worker 8 stopped. I0705 02:13:37.656886 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 53.90063ms to process 0 records I0705 02:13:37.662338 1 tasks_processing.go:74] worker 10 stopped. E0705 02:13:37.662356 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0705 02:13:37.662370 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0705 02:13:37.662376 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0705 02:13:37.662389 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0705 02:13:37.662414 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=2b7e7d968c1ca0e033f2ae1514829fefa6008cf7d89addbe266d351cfb9cca19 I0705 02:13:37.662423 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0705 02:13:37.662427 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0705 02:13:37.662431 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0705 02:13:37.662486 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0705 02:13:37.662496 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0705 02:13:37.662504 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 81.640449ms to process 7 records E0705 02:13:37.666134 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2779ba66d8-d237-4fcb-a4cf-f466f3c79e0f%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.11:32818->172.30.0.10:53: read: connection refused I0705 02:13:37.666149 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%2779ba66d8-d237-4fcb-a4cf-f466f3c79e0f%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.11:32818->172.30.0.10:53: read: connection refused I0705 02:13:37.729271 1 tasks_processing.go:74] worker 28 stopped. I0705 02:13:37.729302 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0705 02:13:37.729311 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 127.718921ms to process 1 records I0705 02:13:37.739692 1 base_controller.go:82] Caches are synced for LoggingSyncer I0705 02:13:37.739705 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0705 02:13:37.745228 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0705 02:13:37.760137 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0705 02:13:37.761320 1 gather_logs.go:145] no pods in namespace were found I0705 02:13:37.761335 1 tasks_processing.go:74] worker 24 stopped. I0705 02:13:37.761345 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 157.067698ms to process 0 records I0705 02:13:37.761374 1 tasks_processing.go:74] worker 15 stopped. I0705 02:13:37.761813 1 recorder.go:75] Recording config/version with fingerprint=ee63618abb85a951402e5758971f3ebd8d0f0d0da1cbe028477da58deee094ef I0705 02:13:37.761869 1 recorder.go:75] Recording config/id with fingerprint=a020276690ce033aa36a23082028034e6faaf47108ea034f764336217e40f076 I0705 02:13:37.761902 1 gather.go:177] gatherer "clusterconfig" function "version" took 160.318383ms to process 2 records W0705 02:13:37.765174 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.11:59547->172.30.0.10:53: read: connection refused I0705 02:13:37.765187 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.129.0.11:59547->172.30.0.10:53: read: connection refused I0705 02:13:37.766692 1 tasks_processing.go:74] worker 3 stopped. E0705 02:13:37.766746 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0705 02:13:37.766776 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rbmoiamflni4jkskg2bjun3635bj0tp-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rbmoiamflni4jkskg2bjun3635bj0tp-primary-cert-bundle-secret" not found I0705 02:13:37.766863 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=c35381c60d0449f06c12e06ae944097af80673ceb2e2be554621cc187a82375d I0705 02:13:37.766906 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 186.400777ms to process 1 records I0705 02:13:37.800483 1 tasks_processing.go:74] worker 25 stopped. I0705 02:13:37.800502 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 199.002544ms to process 0 records W0705 02:13:38.607265 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 02:13:38.817542 1 gather_cluster_operator_pods_and_events.go:121] Found 17 pods with 20 containers I0705 02:13:38.817556 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1258291 bytes I0705 02:13:38.817585 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-bg879 pod in namespace openshift-dns (previous: false). I0705 02:13:39.052853 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0705 02:13:39.075244 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-bg879 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-bg879\" is waiting to start: ContainerCreating" I0705 02:13:39.075270 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-bg879\" is waiting to start: ContainerCreating" I0705 02:13:39.075281 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-bg879 pod in namespace openshift-dns (previous: false). I0705 02:13:39.221665 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-bg879 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-bg879\" is waiting to start: ContainerCreating" I0705 02:13:39.221686 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-bg879\" is waiting to start: ContainerCreating" I0705 02:13:39.221702 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-tq9cj pod in namespace openshift-dns (previous: false). I0705 02:13:39.437613 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-tq9cj pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-tq9cj\" is waiting to start: ContainerCreating" I0705 02:13:39.437635 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-tq9cj\" is waiting to start: ContainerCreating" I0705 02:13:39.437646 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-tq9cj pod in namespace openshift-dns (previous: false). W0705 02:13:39.607862 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 02:13:39.617451 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-tq9cj pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-tq9cj\" is waiting to start: ContainerCreating" I0705 02:13:39.617471 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-tq9cj\" is waiting to start: ContainerCreating" I0705 02:13:39.617485 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-hfb28 pod in namespace openshift-dns (previous: false). I0705 02:13:39.652891 1 tasks_processing.go:74] worker 16 stopped. I0705 02:13:39.652949 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=30f13cf5c8fb741ab9a8c2f13945ddcbcdd9d59ce883e25cf7b2c8b9b53b0d3f I0705 02:13:39.652989 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=64b43e6f4b58aa6eb88dd8bc9f41d72c048e6c3c1829abd3d5dec6676cfea259 I0705 02:13:39.653030 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0705 02:13:39.653070 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=bc9ed599d71f42a07048b0ec2e9e88bf1969bb2f2da285e1dddd4383520abf2c I0705 02:13:39.653091 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0705 02:13:39.653135 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=6fffe329f39c10c496e7b44b421ba406c895a11f6a9fd8231b3d8d4b294f9215 I0705 02:13:39.653167 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=b8c7bb0dff899b97edf10886b3b8c0592dd4a6e0dc46143f01bb90c4d70aa578 I0705 02:13:39.653191 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=b23e7de8f539b9dbe90449235468d731ffb8678e9659ae461adae9919d0ab6af I0705 02:13:39.653212 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=0578f46293be5e7270ab287ac783356ece6bb9f0c988be6e145e2d85b4e644ce I0705 02:13:39.653237 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=adc9e91df91540d7695e53009828dbb662fb26218ea51b9d63a0ac4a52fc711a I0705 02:13:39.653254 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0705 02:13:39.653280 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=d2e9fe6c25c7c024c419bc295eea343edc25c62a09d7ad093051fd49229beafd I0705 02:13:39.653291 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0705 02:13:39.653312 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=6e32ecbb81ac44ee4772a6686c2f6e319ac5277ec7f362b2e67d3fa5f435003f I0705 02:13:39.653322 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0705 02:13:39.653342 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=4d8cec6b84d6f6089c6187192d44be434281f12d72251d9a7e095f21552d0df3 I0705 02:13:39.653351 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0705 02:13:39.653366 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=c55a1852883e4dc59f7acdcfc0c970db2a394ccce9a459784b693271bd48fb2f I0705 02:13:39.653509 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=1b6bf9fa7048e260a772977860613bc2bce8c20999523baead806b85d934bd12 I0705 02:13:39.653524 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0705 02:13:39.653534 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0705 02:13:39.653561 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0705 02:13:39.653583 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=d3fc9ae4bcc5bed67983f7960516e086aeef80271067f17c0747a9314de2df69 I0705 02:13:39.653630 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=e5046504fe9786ff130237518dfcead9695a49a0ac03809e0973d5f053da9cf5 I0705 02:13:39.653650 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0705 02:13:39.653679 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=7584d26ea8733279f5410c8caa6d89babbfe69cbb51cfcda1a027c2b61883cc0 I0705 02:13:39.653692 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0705 02:13:39.653706 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=414cc65ef6582237e0845aa1c4a18807b51e901bf27c0d5994431b81ce93d283 I0705 02:13:39.653718 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=95c381554f99274ba64651902cfafde0c6b104231c556502a0ed36ea1cb3833a I0705 02:13:39.653731 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=517badcaeb1d74b85e7fef90525bf0ebd18abef62c327cc2efee40c967867691 I0705 02:13:39.653745 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=2851d826d66d1c4c79484a93f6f0ab69049d8846fe75421583577590904eccff I0705 02:13:39.653772 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=c4172e2f9c565f814e9f51f523cb7185a5e926ada2f7b6d46e655c94155a2662 I0705 02:13:39.653782 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0705 02:13:39.653809 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=f69167e9d64f13e7222c0b3fc5f3ed72f46c9de8c75127cd07c791522711e3b1 I0705 02:13:39.653825 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0705 02:13:39.653832 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0705 02:13:39.653839 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.071067859s to process 36 records I0705 02:13:39.816401 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 02:13:39.816416 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-pt5tg pod in namespace openshift-dns (previous: false). I0705 02:13:40.017438 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 02:13:40.017458 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-zcprj pod in namespace openshift-dns (previous: false). I0705 02:13:40.239424 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for node-resolver-zcprj pod in namespace openshift-dns for failing operator dns-node-resolver (previous: false): "container \"dns-node-resolver\" in pod \"node-resolver-zcprj\" is waiting to start: ContainerCreating" I0705 02:13:40.239442 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns-node-resolver\" in pod \"node-resolver-zcprj\" is waiting to start: ContainerCreating" I0705 02:13:40.239453 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-697cd9f6c-kbl7b pod in namespace openshift-image-registry (previous: false). I0705 02:13:40.414729 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 02:13:40.414749 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7896bb8d6d-4ct8h pod in namespace openshift-image-registry (previous: false). W0705 02:13:40.606546 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 02:13:40.618277 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7896bb8d6d-4ct8h pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7896bb8d6d-4ct8h\" is waiting to start: ContainerCreating" I0705 02:13:40.618291 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7896bb8d6d-4ct8h\" is waiting to start: ContainerCreating" I0705 02:13:40.618301 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-7896bb8d6d-kgj2n pod in namespace openshift-image-registry (previous: false). I0705 02:13:40.825217 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-7896bb8d6d-kgj2n pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-7896bb8d6d-kgj2n\" is waiting to start: ContainerCreating" I0705 02:13:40.825233 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-7896bb8d6d-kgj2n\" is waiting to start: ContainerCreating" I0705 02:13:40.825242 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-mjjhx pod in namespace openshift-image-registry (previous: false). I0705 02:13:41.017568 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 02:13:41.017584 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-t68sm pod in namespace openshift-image-registry (previous: false). I0705 02:13:41.221524 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for node-ca-t68sm pod in namespace openshift-image-registry for failing operator node-ca (previous: false): "container \"node-ca\" in pod \"node-ca-t68sm\" is waiting to start: ContainerCreating" I0705 02:13:41.221546 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"node-ca\" in pod \"node-ca-t68sm\" is waiting to start: ContainerCreating" I0705 02:13:41.221559 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-z5w2z pod in namespace openshift-image-registry (previous: false). I0705 02:13:41.421276 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0705 02:13:41.421310 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7db758d579-4bscw pod in namespace openshift-ingress (previous: false). W0705 02:13:41.606783 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0705 02:13:41.617821 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7db758d579-4bscw pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7db758d579-4bscw\" is waiting to start: ContainerCreating" I0705 02:13:41.617839 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7db758d579-4bscw\" is waiting to start: ContainerCreating" I0705 02:13:41.617852 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-7db758d579-hpm5j pod in namespace openshift-ingress (previous: false). I0705 02:13:41.828291 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-7db758d579-hpm5j pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-7db758d579-hpm5j\" is waiting to start: ContainerCreating" I0705 02:13:41.828309 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-7db758d579-hpm5j\" is waiting to start: ContainerCreating" I0705 02:13:41.828322 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-7djxf pod in namespace openshift-ingress-canary (previous: false). I0705 02:13:42.048358 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-7djxf pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-7djxf\" is waiting to start: ContainerCreating" I0705 02:13:42.048380 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-7djxf\" is waiting to start: ContainerCreating" I0705 02:13:42.048393 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-tlpwk pod in namespace openshift-ingress-canary (previous: false). I0705 02:13:42.222712 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-tlpwk pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-tlpwk\" is waiting to start: ContainerCreating" I0705 02:13:42.222761 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-tlpwk\" is waiting to start: ContainerCreating" I0705 02:13:42.222797 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-bl6m4 pod in namespace openshift-kube-storage-version-migrator (previous: false). I0705 02:13:42.421857 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-bl6m4 pod in namespace openshift-kube-storage-version-migrator (previous: false). W0705 02:13:42.607432 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0705 02:13:42.607456 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0705 02:13:42.607470 1 tasks_processing.go:74] worker 1 stopped. E0705 02:13:42.607479 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0705 02:13:42.607489 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0705 02:13:42.607503 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0705 02:13:42.607512 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.027471318s to process 1 records I0705 02:13:42.617727 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-dhm84 pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0705 02:13:42.819969 1 tasks_processing.go:74] worker 2 stopped. I0705 02:13:42.820072 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=b939b8fcf58187bf61a6a9f36012085ae2d7341243724c812770601be5786ead I0705 02:13:42.820126 1 recorder.go:75] Recording events/openshift-dns with fingerprint=5554ed307109b7b85fecaf24920cfb9612186c604b4f54931b48d031fae70f8f I0705 02:13:42.820222 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=664f9ceef73fb702cbe69f9cd03799a469a1eb9bd37f7b2bb45348932c3594bd I0705 02:13:42.820268 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=18d230ef76b533bfc58a09320d8a13b43a4d20f66b9640f190f98f9d35e0e6d9 I0705 02:13:42.820321 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=3f40d2c1112de55db60fa11ac39242080d850bbbfd95f0b5a71c4199d2ff3143 I0705 02:13:42.820343 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=c76171eebba586d40981479c337d9658d05ed79e02f52a9672b4b7be0b3df999 I0705 02:13:42.820361 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=3e786037a547e140bc2d684b1ebb9602d5117353e9ff4d33fa8bd29de92b3f15 I0705 02:13:42.820414 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=ef5ccffca6c42b91c04cfe777cf985d392c007d12f10ef6ac4c187ef7b9209bb I0705 02:13:42.820437 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-bl6m4/migrator_current.log with fingerprint=f750b840fb87b6541d9fc92225e3541bb69849aa9eb29100d7b3a8ab56791534 I0705 02:13:42.820446 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-bl6m4/graceful-termination_current.log with fingerprint=0e736ada1b357b93a3e9451c8165a50063994d11abb0c5083549fd8ee0f49d4f I0705 02:13:42.820536 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-dhm84/kube-storage-version-migrator-operator_current.log with fingerprint=84b547f635501897a776e19eda1ddf2e6a64deedb13b83536f5e223d26c2dd0f I0705 02:13:42.820549 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 5.239868787s to process 11 records I0705 02:13:49.489341 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:13:50.610775 1 tasks_processing.go:74] worker 4 stopped. I0705 02:13:50.610811 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0705 02:13:50.610824 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.030451111s to process 1 records I0705 02:13:50.677989 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:13:50.877111 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:13:51.387658 1 tasks_processing.go:74] worker 26 stopped. I0705 02:13:51.387930 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=3a23a80037b37537e6bce19267a0686640cc3d2d84bcc6f7bc973979a947d17d I0705 02:13:51.387946 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.80550111s to process 1 records E0705 02:13:51.388010 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.808s with: function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0705 02:13:51.389111 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0705 02:13:51.389127 1 periodic.go:209] Running workloads gatherer I0705 02:13:51.389143 1 tasks_processing.go:45] number of workers: 2 I0705 02:13:51.389148 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 02:13:51.389153 1 tasks_processing.go:71] worker 1 working on workload_info task. I0705 02:13:51.389162 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 02:13:51.389174 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0705 02:13:51.411414 1 tasks_processing.go:74] worker 0 stopped. I0705 02:13:51.411435 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 22.225352ms to process 0 records I0705 02:13:51.413067 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0705 02:13:51.420053 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0705 02:13:51.429767 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (10ms) I0705 02:13:51.437635 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (8ms) I0705 02:13:51.448365 1 gather_workloads_info.go:387] No image sha256:eff3318d5ac1d3a141221b536233305f54b3334fab56e5f8ba20ef8a9ed21429 (11ms) I0705 02:13:51.456730 1 gather_workloads_info.go:387] No image sha256:80748ba08e1c264a8c105e7f607eff386a66378e024443a844993ee9292858c1 (8ms) I0705 02:13:51.463722 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (7ms) I0705 02:13:51.473747 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (10ms) I0705 02:13:51.480539 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (7ms) I0705 02:13:51.487820 1 gather_workloads_info.go:387] No image sha256:fac526ab9335a9a8decb028a30673f4ae718deae8da114f36103b6438ba6458f (7ms) I0705 02:13:51.494457 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (7ms) I0705 02:13:51.519898 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (25ms) I0705 02:13:51.620639 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (101ms) I0705 02:13:51.721603 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (101ms) I0705 02:13:51.820911 1 gather_workloads_info.go:387] No image sha256:d3209a996e5ce362136e70a0bcf06d89973905de73e234b644d8ae8886d067f1 (99ms) I0705 02:13:51.921236 1 gather_workloads_info.go:387] No image sha256:ef2bfb546cd8aac31b7671585f718f44b268ed43a9c8562016ab741558a7dfa1 (100ms) I0705 02:13:52.020705 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (99ms) I0705 02:13:52.121662 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (101ms) I0705 02:13:52.221327 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0705 02:13:52.320855 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0705 02:13:52.421304 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0705 02:13:52.520918 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (100ms) I0705 02:13:52.620670 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (100ms) I0705 02:13:52.724377 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (104ms) I0705 02:13:52.822980 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (99ms) I0705 02:13:52.922413 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (99ms) I0705 02:13:53.023265 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (101ms) I0705 02:13:53.121831 1 gather_workloads_info.go:387] No image sha256:da4f8a01ef5d0607cf76fa7b219fafdb6020cb4cf9a0a88cfe76d292c1a561fc (98ms) I0705 02:13:53.222257 1 gather_workloads_info.go:387] No image sha256:a505dd5b62d744c2cfda2666b5e21e0a4704354a3114488bd3291456085e1e33 (100ms) I0705 02:13:53.320884 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (99ms) I0705 02:13:53.425124 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (104ms) I0705 02:13:53.520946 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (96ms) I0705 02:13:53.620936 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0705 02:13:53.722241 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0705 02:13:53.823068 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (101ms) I0705 02:13:53.922431 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (99ms) I0705 02:13:54.021418 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (99ms) I0705 02:13:54.122071 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (101ms) I0705 02:13:54.222005 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0705 02:13:54.321441 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (99ms) I0705 02:13:54.421607 1 gather_workloads_info.go:387] No image sha256:fab570ecdc9c3620b032bc4c5b44240f5ebac0bb9e48dfeea32bcd36eb462e5a (100ms) I0705 02:13:54.421656 1 tasks_processing.go:74] worker 1 stopped. E0705 02:13:54.421670 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0705 02:13:54.422153 1 recorder.go:75] Recording config/workload_info with fingerprint=5c61ea0aec2c08313be532538cc4cd4e0711ba174723c8bedbec5b55e07a7238 I0705 02:13:54.422175 1 gather.go:177] gatherer "workloads" function "workload_info" took 3.032495536s to process 1 records E0705 02:13:54.422207 1 periodic.go:247] "Unhandled Error" err="workloads failed after 3.033s with: function \"workload_info\" failed with an error" I0705 02:13:54.423331 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0705 02:13:54.423347 1 periodic.go:209] Running conditional gatherer I0705 02:13:54.429837 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0705 02:13:54.440305 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.11:46906->172.30.0.10:53: read: connection refused E0705 02:13:54.440729 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0705 02:13:54.440854 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0705 02:13:54.452611 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0705 02:13:54.452630 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452638 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452642 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452648 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452652 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452657 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452662 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452667 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0705 02:13:54.452671 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0705 02:13:54.452684 1 tasks_processing.go:45] number of workers: 3 I0705 02:13:54.452693 1 tasks_processing.go:69] worker 2 listening for tasks. I0705 02:13:54.452699 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0705 02:13:54.452813 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0705 02:13:54.452872 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.016µs to process 1 records I0705 02:13:54.452903 1 tasks_processing.go:69] worker 0 listening for tasks. I0705 02:13:54.452929 1 tasks_processing.go:69] worker 1 listening for tasks. I0705 02:13:54.452941 1 tasks_processing.go:74] worker 1 stopped. I0705 02:13:54.452947 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0705 02:13:54.453283 1 tasks_processing.go:74] worker 2 stopped. I0705 02:13:54.453327 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 329.666µs to process 0 records I0705 02:13:54.453357 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0705 02:13:54.453383 1 tasks_processing.go:74] worker 0 stopped. I0705 02:13:54.453461 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0705 02:13:54.453495 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 922ns to process 1 records I0705 02:13:54.453546 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.129.0.11:46906->172.30.0.10:53: read: connection refused I0705 02:13:54.453604 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0705 02:13:54.473670 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=903a4430f45dc0bc109b3a1b806c3b94eca60e1ed4ffff956d73c29c811d1f03 I0705 02:13:54.473844 1 diskrecorder.go:70] Writing 106 records to /var/lib/insights-operator/insights-2026-07-05-021354.tar.gz I0705 02:13:54.483265 1 diskrecorder.go:51] Wrote 106 records to disk in 9ms I0705 02:13:54.483335 1 periodic.go:278] Gathering cluster info every 2h0m0s I0705 02:13:54.483377 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0705 02:14:00.886177 1 configmapobserver.go:84] configmaps "insights-config" not found I0705 02:14:41.274062 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="16b5497b472199cd0c3fb7b5c1baa3472d36f4cab5baf80af3fa011abd8caeb2") W0705 02:14:41.274091 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0705 02:14:41.274148 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="db2d296ff9f7c82f7cb2d517084da16fa51735136bd28a4389a69ec73ef35732") I0705 02:14:41.274167 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0705 02:14:41.274185 1 periodic.go:170] Shutting down I0705 02:14:41.274191 1 base_controller.go:181] Shutting down ConfigController ... I0705 02:14:41.274207 1 base_controller.go:181] Shutting down LoggingSyncer ... I0705 02:14:41.274217 1 base_controller.go:123] Shutting down worker of ConfigController controller ... I0705 02:14:41.274227 1 base_controller.go:113] All ConfigController workers have been terminated