W0706 01:08:25.745489 1 cmd.go:257] Using insecure, self-signed certificates I0706 01:08:26.505465 1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0706 01:08:26.505810 1 observer_polling.go:159] Starting file observer I0706 01:08:27.053655 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0706 01:08:27.053929 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0706 01:08:27.054826 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0706 01:08:27.054911 1 secure_serving.go:57] Forcing use of http/1.1 only W0706 01:08:27.054937 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0706 01:08:27.054944 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0706 01:08:27.054950 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0706 01:08:27.054953 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0706 01:08:27.054957 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0706 01:08:27.054960 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0706 01:08:27.060945 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0706 01:08:27.060965 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0706 01:08:27.060970 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0706 01:08:27.060973 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0706 01:08:27.060991 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0706 01:08:27.060995 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0706 01:08:27.061284 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-827950272/tls.crt::/tmp/serving-cert-827950272/tls.key" I0706 01:08:27.062078 1 secure_serving.go:213] Serving securely on [::]:8443 I0706 01:08:27.062368 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" I0706 01:08:27.062332 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0706 01:08:27.062383 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"0bc5421d-1043-48d1-bf6e-4b0dede6f87f", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} W0706 01:08:27.072063 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0706 01:08:27.072094 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0706 01:08:27.072202 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0706 01:08:27.084296 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0706 01:08:27.084326 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0706 01:08:27.095955 1 secretconfigobserver.go:119] support secret does not exist I0706 01:08:27.106305 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0706 01:08:27.118076 1 secretconfigobserver.go:119] support secret does not exist I0706 01:08:27.118341 1 recorder.go:161] Pruning old reports every 7h43m14s, max age is 288h0m0s I0706 01:08:27.127153 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0706 01:08:27.127166 1 periodic.go:209] Running clusterconfig gatherer I0706 01:08:27.127176 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0706 01:08:27.127200 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0706 01:08:27.127208 1 insightsreport.go:296] Starting report retriever I0706 01:08:27.127214 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0706 01:08:27.127221 1 tasks_processing.go:45] number of workers: 32 I0706 01:08:27.127246 1 tasks_processing.go:69] worker 2 listening for tasks. I0706 01:08:27.127260 1 tasks_processing.go:69] worker 1 listening for tasks. I0706 01:08:27.127260 1 tasks_processing.go:71] worker 2 working on silenced_alerts task. I0706 01:08:27.127266 1 tasks_processing.go:71] worker 1 working on lokistack task. I0706 01:08:27.127261 1 tasks_processing.go:69] worker 0 listening for tasks. I0706 01:08:27.127268 1 tasks_processing.go:69] worker 3 listening for tasks. I0706 01:08:27.127275 1 tasks_processing.go:69] worker 4 listening for tasks. I0706 01:08:27.127282 1 tasks_processing.go:69] worker 17 listening for tasks. I0706 01:08:27.127284 1 tasks_processing.go:69] worker 9 listening for tasks. I0706 01:08:27.127282 1 tasks_processing.go:69] worker 22 listening for tasks. I0706 01:08:27.127259 1 tasks_processing.go:69] worker 12 listening for tasks. I0706 01:08:27.127301 1 tasks_processing.go:69] worker 14 listening for tasks. I0706 01:08:27.127294 1 tasks_processing.go:69] worker 13 listening for tasks. I0706 01:08:27.127301 1 tasks_processing.go:69] worker 11 listening for tasks. I0706 01:08:27.127308 1 tasks_processing.go:69] worker 8 listening for tasks. I0706 01:08:27.127315 1 tasks_processing.go:69] worker 24 listening for tasks. W0706 01:08:27.127295 1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0706 01:08:27.127316 1 tasks_processing.go:69] worker 16 listening for tasks. I0706 01:08:27.127308 1 tasks_processing.go:69] worker 18 listening for tasks. I0706 01:08:27.127323 1 tasks_processing.go:69] worker 20 listening for tasks. I0706 01:08:27.127325 1 tasks_processing.go:69] worker 21 listening for tasks. I0706 01:08:27.127295 1 tasks_processing.go:69] worker 10 listening for tasks. I0706 01:08:27.127331 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 54.836µs to process 0 records I0706 01:08:27.127336 1 tasks_processing.go:69] worker 28 listening for tasks. I0706 01:08:27.127336 1 tasks_processing.go:69] worker 26 listening for tasks. I0706 01:08:27.127341 1 tasks_processing.go:69] worker 27 listening for tasks. I0706 01:08:27.127340 1 tasks_processing.go:69] worker 30 listening for tasks. I0706 01:08:27.127345 1 tasks_processing.go:69] worker 31 listening for tasks. I0706 01:08:27.127310 1 tasks_processing.go:69] worker 15 listening for tasks. I0706 01:08:27.127347 1 tasks_processing.go:69] worker 7 listening for tasks. I0706 01:08:27.127315 1 tasks_processing.go:69] worker 19 listening for tasks. I0706 01:08:27.127306 1 tasks_processing.go:69] worker 23 listening for tasks. I0706 01:08:27.127325 1 tasks_processing.go:69] worker 5 listening for tasks. I0706 01:08:27.127327 1 tasks_processing.go:69] worker 25 listening for tasks. I0706 01:08:27.127333 1 tasks_processing.go:69] worker 29 listening for tasks. I0706 01:08:27.127335 1 tasks_processing.go:69] worker 6 listening for tasks. I0706 01:08:27.127416 1 tasks_processing.go:71] worker 12 working on aggregated_monitoring_cr_names task. I0706 01:08:27.127423 1 tasks_processing.go:71] worker 0 working on feature_gates task. I0706 01:08:27.127426 1 tasks_processing.go:71] worker 4 working on nodes task. I0706 01:08:27.127433 1 tasks_processing.go:71] worker 11 working on config_maps task. I0706 01:08:27.127438 1 tasks_processing.go:71] worker 3 working on certificate_signing_requests task. I0706 01:08:27.127453 1 tasks_processing.go:71] worker 26 working on storage_classes task. I0706 01:08:27.127502 1 tasks_processing.go:71] worker 27 working on jaegers task. I0706 01:08:27.127542 1 tasks_processing.go:71] worker 7 working on container_runtime_configs task. I0706 01:08:27.127425 1 tasks_processing.go:71] worker 14 working on install_plans task. I0706 01:08:27.127429 1 tasks_processing.go:71] worker 13 working on overlapping_namespace_uids task. I0706 01:08:27.127509 1 tasks_processing.go:71] worker 8 working on image_registries task. I0706 01:08:27.127510 1 tasks_processing.go:71] worker 9 working on tsdb_status task. W0706 01:08:27.128115 1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0706 01:08:27.127512 1 tasks_processing.go:71] worker 24 working on crds task. I0706 01:08:27.128208 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 94.08µs to process 0 records I0706 01:08:27.127515 1 tasks_processing.go:71] worker 2 working on openshift_logging task. I0706 01:08:27.127519 1 tasks_processing.go:71] worker 16 working on cluster_apiserver task. I0706 01:08:27.127515 1 tasks_processing.go:71] worker 17 working on sap_datahubs task. I0706 01:08:27.127523 1 tasks_processing.go:71] worker 18 working on node_logs task. I0706 01:08:27.127525 1 tasks_processing.go:71] worker 19 working on operators task. I0706 01:08:27.127527 1 tasks_processing.go:71] worker 20 working on mutating_webhook_configurations task. I0706 01:08:27.127530 1 tasks_processing.go:71] worker 21 working on openstack_version task. I0706 01:08:27.127530 1 tasks_processing.go:71] worker 30 working on monitoring_persistent_volumes task. I0706 01:08:27.127534 1 tasks_processing.go:71] worker 10 working on openshift_machine_api_events task. I0706 01:08:27.127535 1 tasks_processing.go:71] worker 31 working on oauths task. I0706 01:08:27.127538 1 tasks_processing.go:71] worker 15 working on infrastructures task. I0706 01:08:27.127537 1 tasks_processing.go:71] worker 28 working on nodenetworkstates task. I0706 01:08:27.127547 1 tasks_processing.go:71] worker 22 working on support_secret task. I0706 01:08:27.127557 1 tasks_processing.go:71] worker 5 working on machine_autoscalers task. I0706 01:08:27.127560 1 tasks_processing.go:71] worker 23 working on machine_configs task. I0706 01:08:27.127570 1 tasks_processing.go:71] worker 25 working on ingress_certificates task. I0706 01:08:27.127577 1 tasks_processing.go:71] worker 29 working on nodenetworkconfigurationpolicies task. I0706 01:08:27.127420 1 tasks_processing.go:71] worker 6 working on openstack_dataplanedeployments task. I0706 01:08:27.128195 1 tasks_processing.go:71] worker 9 working on machine_config_pools task. I0706 01:08:27.136909 1 tasks_processing.go:71] worker 1 working on openstack_controlplanes task. I0706 01:08:27.137014 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 9.634346ms to process 0 records I0706 01:08:27.137030 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 9.361392ms to process 0 records I0706 01:08:27.137039 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 9.445598ms to process 0 records I0706 01:08:27.137045 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 8.040998ms to process 0 records I0706 01:08:27.137075 1 tasks_processing.go:71] worker 27 working on service_accounts task. I0706 01:08:27.137080 1 tasks_processing.go:71] worker 17 working on dvo_metrics task. I0706 01:08:27.137264 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=52b9230d91a0b27787a3bb485a3da3ea2e1d26937fff7e1a0c3c5c4386b9f9b1 I0706 01:08:27.137089 1 tasks_processing.go:71] worker 7 working on machine_healthchecks task. I0706 01:08:27.137291 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=dfa2d564903257a2f111712abb9eef1eca7114442ed7faa474c0f13d8f04543a I0706 01:08:27.137308 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 9.633985ms to process 2 records I0706 01:08:27.137318 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 8.765648ms to process 0 records I0706 01:08:27.137327 1 tasks_processing.go:71] worker 2 working on machine_sets task. I0706 01:08:27.137546 1 tasks_processing.go:71] worker 0 working on pod_network_connectivity_checks task. I0706 01:08:27.137097 1 tasks_processing.go:71] worker 26 working on openstack_dataplanenodesets task. I0706 01:08:27.137671 1 recorder.go:75] Recording config/featuregate with fingerprint=e8532f737cf97b5eba5f54f552fb43bda21e44e117dc4bbbd28b7c06a2a44a05 I0706 01:08:27.137693 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 10.047521ms to process 1 records I0706 01:08:27.137212 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0706 01:08:27.137757 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0706 01:08:27.137765 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0706 01:08:27.137771 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0706 01:08:27.137793 1 controller.go:489] The operator is still being initialized I0706 01:08:27.137805 1 controller.go:512] The operator is healthy I0706 01:08:27.139426 1 tasks_processing.go:71] worker 3 working on machines task. I0706 01:08:27.139440 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 11.972426ms to process 0 records I0706 01:08:27.141935 1 tasks_processing.go:71] worker 16 working on container_images task. I0706 01:08:27.142160 1 recorder.go:75] Recording config/apiserver with fingerprint=425e0c20b41bb666f6c2e15d07836cc40c23df690d56dd874aa7c097576ae44a I0706 01:08:27.142190 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 13.508775ms to process 1 records I0706 01:08:27.148475 1 tasks_processing.go:71] worker 21 working on networks task. I0706 01:08:27.148493 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 18.717413ms to process 0 records I0706 01:08:27.148517 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 16.781159ms to process 0 records I0706 01:08:27.148523 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 16.686111ms to process 0 records I0706 01:08:27.148529 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 18.269497ms to process 0 records I0706 01:08:27.148537 1 tasks_processing.go:71] worker 10 working on cost_management_metrics_configs task. I0706 01:08:27.148669 1 tasks_processing.go:71] worker 29 working on olm_operators task. I0706 01:08:27.148759 1 tasks_processing.go:71] worker 6 working on validating_webhook_configurations task. I0706 01:08:27.148923 1 tasks_processing.go:71] worker 31 working on ceph_cluster task. I0706 01:08:27.149188 1 recorder.go:75] Recording config/oauth with fingerprint=bbcdd27e695f8304ecdbfb7d493118f1311aaa57981f4e34ba2c5d54ecc8f61e I0706 01:08:27.149208 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 18.432175ms to process 1 records I0706 01:08:27.149373 1 tasks_processing.go:71] worker 5 working on active_alerts task. I0706 01:08:27.149389 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 18.133758ms to process 0 records I0706 01:08:27.149400 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 20.165994ms to process 0 records W0706 01:08:27.149406 1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0706 01:08:27.149411 1 tasks_processing.go:71] worker 18 working on image_pruners task. I0706 01:08:27.149419 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 28.761µs to process 0 records I0706 01:08:27.149430 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 18.541399ms to process 0 records I0706 01:08:27.149483 1 tasks_processing.go:71] worker 28 working on storage_cluster task. I0706 01:08:27.149507 1 tasks_processing.go:71] worker 5 working on operators_pods_and_events task. I0706 01:08:27.150214 1 tasks_processing.go:71] worker 4 working on sap_pods task. I0706 01:08:27.151774 1 recorder.go:75] Recording config/node/ip-10-0-0-67.ec2.internal with fingerprint=69ea0646c60304c433a0416603939cd65f1115d34c1b44898c43e1ba01ff451b I0706 01:08:27.152269 1 recorder.go:75] Recording config/node/ip-10-0-1-126.ec2.internal with fingerprint=0ea91a482582a13562064ad4a087593e6982b82adce90606aeab68e3e4867883 I0706 01:08:27.152458 1 recorder.go:75] Recording config/node/ip-10-0-2-32.ec2.internal with fingerprint=9402b998e5f8f77cff310d7cdc2ed40cd8b0abc37fe45beb4973ef112d7bf35a I0706 01:08:27.152502 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 22.77485ms to process 3 records I0706 01:08:27.153556 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=7246fff3f0d4091041f697c6bcc60d2efd0d5b53dbfd4a3f6eab46ade234dae0 I0706 01:08:27.153578 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 23.853848ms to process 1 records I0706 01:08:27.153611 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 16.309966ms to process 0 records I0706 01:08:27.153625 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 15.924811ms to process 0 records E0706 01:08:27.153634 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0706 01:08:27.153641 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 15.711546ms to process 0 records I0706 01:08:27.153835 1 tasks_processing.go:71] worker 1 working on authentication task. I0706 01:08:27.154121 1 tasks_processing.go:71] worker 8 working on proxies task. I0706 01:08:27.154183 1 tasks_processing.go:71] worker 2 working on qemu_kubevirt_launcher_logs task. I0706 01:08:27.154203 1 tasks_processing.go:71] worker 0 working on image task. I0706 01:08:27.154186 1 tasks_processing.go:71] worker 15 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0706 01:08:27.155935 1 recorder.go:75] Recording config/infrastructure with fingerprint=5fca775c0560c636680e13399407143016f776da9a02a5243eaa2a213820ff2e I0706 01:08:27.155966 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 22.880224ms to process 1 records I0706 01:08:27.155983 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 16.002077ms to process 0 records I0706 01:08:27.155999 1 tasks_processing.go:71] worker 26 working on pdbs task. I0706 01:08:27.156012 1 tasks_processing.go:71] worker 13 working on sap_config task. I0706 01:08:27.156004 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0706 01:08:27.156116 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 25.895547ms to process 1 records I0706 01:08:27.158718 1 tasks_processing.go:71] worker 30 working on ingress task. I0706 01:08:27.158777 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 28.727344ms to process 0 records I0706 01:08:27.158817 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 9.824656ms to process 0 records I0706 01:08:27.158843 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 26.678776ms to process 0 records I0706 01:08:27.158847 1 tasks_processing.go:71] worker 31 working on clusterroles task. E0706 01:08:27.158861 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0706 01:08:27.158866 1 tasks_processing.go:71] worker 22 working on version task. I0706 01:08:27.158857 1 tasks_processing.go:71] worker 9 working on metrics task. I0706 01:08:27.158869 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 27.709592ms to process 0 records E0706 01:08:27.158891 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0706 01:08:27.158897 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 21.583955ms to process 0 records W0706 01:08:27.158901 1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0706 01:08:27.158907 1 tasks_processing.go:71] worker 7 working on schedulers task. I0706 01:08:27.158915 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 25.523µs to process 0 records I0706 01:08:27.158934 1 tasks_processing.go:74] worker 9 stopped. I0706 01:08:27.159062 1 tasks_processing.go:74] worker 20 stopped. I0706 01:08:27.159248 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=497c4077501c8a48c07d558b3185ad64e07935811faf707ba5da509ccd4e5f2c I0706 01:08:27.159293 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=1de6e8aab360a27b271f249fb4a0c71296ffb09fd1ea33b921832a78c5e47e93 I0706 01:08:27.159332 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=665e459de79dc2bd6aa2c1a011e179f36a5edb3df5c55cc54111560bde312e34 I0706 01:08:27.159340 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 29.447711ms to process 3 records I0706 01:08:27.160503 1 tasks_processing.go:74] worker 28 stopped. I0706 01:08:27.160519 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 11.002488ms to process 0 records E0706 01:08:27.160529 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0706 01:08:27.160535 1 gather.go:177] gatherer "clusterconfig" function "machines" took 21.086599ms to process 0 records I0706 01:08:27.160541 1 tasks_processing.go:74] worker 3 stopped. I0706 01:08:27.160755 1 tasks_processing.go:74] worker 4 stopped. I0706 01:08:27.160771 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 10.52317ms to process 0 records I0706 01:08:27.161799 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0706 01:08:27.161799 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0706 01:08:27.161821 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0706 01:08:27.161839 1 tasks_processing.go:74] worker 10 stopped. I0706 01:08:27.161855 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 13.283044ms to process 0 records I0706 01:08:27.169111 1 tasks_processing.go:74] worker 13 stopped. I0706 01:08:27.169125 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 13.087552ms to process 0 records I0706 01:08:27.169339 1 tasks_processing.go:74] worker 29 stopped. I0706 01:08:27.169358 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 20.644501ms to process 0 records I0706 01:08:27.169792 1 tasks_processing.go:74] worker 24 stopped. I0706 01:08:27.170413 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=22fedcb3d091efaffb775d0133cfb1b587b6e290d610f4426db58d41a90c2d17 I0706 01:08:27.170745 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=9098beb066ecb13d83a1b366124dac263601b4d7977ffc75de462f172459a50d I0706 01:08:27.170762 1 gather.go:177] gatherer "clusterconfig" function "crds" took 41.582139ms to process 2 records I0706 01:08:27.170848 1 tasks_processing.go:74] worker 18 stopped. I0706 01:08:27.170881 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=5c36ef9e6e06d74df22fc79329ed897398acd78d2cf03d0ee1db46b6e8a1eb99 I0706 01:08:27.170896 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 20.50805ms to process 1 records I0706 01:08:27.170906 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 42.553317ms to process 0 records I0706 01:08:27.170926 1 tasks_processing.go:74] worker 12 stopped. I0706 01:08:27.170952 1 recorder.go:75] Recording config/proxy with fingerprint=f03c5cc659cd8ae1fdef8d9048830310d9896ac33678073cc536efebb499eafe I0706 01:08:27.170961 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 15.841361ms to process 1 records I0706 01:08:27.170970 1 tasks_processing.go:74] worker 8 stopped. I0706 01:08:27.172266 1 base_controller.go:82] Caches are synced for ConfigController I0706 01:08:27.172279 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0706 01:08:27.173508 1 tasks_processing.go:74] worker 0 stopped. I0706 01:08:27.173669 1 recorder.go:75] Recording config/image with fingerprint=9a68241fef6f9dd5055d3ee2195521facc3fd1d9abc2cd7e4076eaa5d0790c14 I0706 01:08:27.173688 1 gather.go:177] gatherer "clusterconfig" function "image" took 19.293258ms to process 1 records I0706 01:08:27.173782 1 tasks_processing.go:74] worker 7 stopped. I0706 01:08:27.173807 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=062bedb75478f259793d6d4c5a1fca0c80b2be60a70648d86693e5bb30ed0d4c I0706 01:08:27.173823 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 14.623242ms to process 1 records W0706 01:08:27.173838 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0706 01:08:27.173910 1 tasks_processing.go:74] worker 1 stopped. I0706 01:08:27.174060 1 recorder.go:75] Recording config/authentication with fingerprint=dc7b797ba46936510d3236e266fa5f8cc99adb9fa085f686619ebe7aba864aa1 I0706 01:08:27.174070 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 19.788234ms to process 1 records I0706 01:08:27.174260 1 tasks_processing.go:74] worker 21 stopped. I0706 01:08:27.174273 1 recorder.go:75] Recording config/network with fingerprint=a14dc8b1c76d3bab300cc5d48f17a11d39d201ff436e6c583c52e852a8ec378c I0706 01:08:27.174284 1 gather.go:177] gatherer "clusterconfig" function "networks" took 25.293664ms to process 1 records I0706 01:08:27.174349 1 tasks_processing.go:74] worker 6 stopped. I0706 01:08:27.174571 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=80cc687ac6c2a4fdd7c2fadad55776d827f06aaf90013416b5a2eb06bcfa2ce0 I0706 01:08:27.174827 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0706 01:08:27.174917 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=f844f51219e8847848257f5efa99a867db7d81c37aab727cd01874ca20d35321 I0706 01:08:27.174960 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=1e3757a17ce55e8ca6958ed543080a281a939a4c1eccd8e5930f1f06b7a55c47 I0706 01:08:27.174831 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0706 01:08:27.175053 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=c7c024ed5511af22330254f4853b7d465357d6c8523f29fc08b146c9929cbf33 I0706 01:08:27.175246 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=f1855709418b697607be133a69d1df45484334752869646cfd9efaaa93bd42a7 I0706 01:08:27.175314 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=b245613bb5d604cac1cb356a8e6b2c0fcb0714f24af18e246687e13457debbc3 I0706 01:08:27.175375 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=3c46a358e615efa0d255290e70c51dbd0c9572c1eb57fcbb75e32135f8d78f93 I0706 01:08:27.175468 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=5b7df757628aaf50be0e720451730a7bb70531c7ec94653ff2c654f917c2cd1f I0706 01:08:27.175528 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=59aea705039975027c1b8ab64481cff16a353e2fb2210a00ba7448ac3ec7fb73 W0706 01:08:27.175564 1 operator.go:288] started I0706 01:08:27.175619 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0706 01:08:27.175652 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=76b517defc85a70a9d1cdb2ea6b6f6235e554eaa3303e1fb91da3653085d2a25 I0706 01:08:27.175737 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=cc2791a6a6281ca3bdc5ea919f89c8f2cc6a06751c41d00fa349d88a7217723d I0706 01:08:27.175752 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 25.067613ms to process 11 records I0706 01:08:27.181155 1 gather_logs.go:145] no pods in namespace were found I0706 01:08:27.181181 1 tasks_processing.go:74] worker 2 stopped. I0706 01:08:27.181198 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 26.974785ms to process 0 records I0706 01:08:27.181701 1 tasks_processing.go:74] worker 26 stopped. I0706 01:08:27.181841 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=ff822681ce699f5440bbf38b1aa9d9cc1bf51223e7fc2f606791664860af5bfb I0706 01:08:27.181874 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=c0ef90fdc0300d03119c654b0debcbd57765858025280adb71b484bab5bd3e41 I0706 01:08:27.181898 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=9d17ae54d074987a2c144c9d97e712bce4b11e48fc901dd814ef5c19a4abe0e7 I0706 01:08:27.181909 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 25.688992ms to process 3 records I0706 01:08:27.181996 1 tasks_processing.go:74] worker 30 stopped. I0706 01:08:27.182074 1 recorder.go:75] Recording config/ingress with fingerprint=4a2f23fc024b5bebf06f2e828d7e376e70730ee4a0298725aa64a5fe3d42f612 I0706 01:08:27.182064 1 configmapobserver.go:84] configmaps "insights-config" not found I0706 01:08:27.182085 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 23.166698ms to process 1 records I0706 01:08:27.185004 1 tasks_processing.go:74] worker 23 stopped. I0706 01:08:27.185038 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0706 01:08:27.185050 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 53.475343ms to process 1 records I0706 01:08:27.194355 1 tasks_processing.go:74] worker 22 stopped. I0706 01:08:27.194480 1 prometheus_rules.go:88] Prometheus rules successfully created I0706 01:08:27.194889 1 recorder.go:75] Recording config/version with fingerprint=93da96db359905f6d3687f62a086f531c1d6cd83f619483d3180a7e4388cc515 I0706 01:08:27.194907 1 recorder.go:75] Recording config/id with fingerprint=3500a8d396364d2c6a66a9ebd09be3ab5b3aa815f555e195068860612701416a I0706 01:08:27.194916 1 gather.go:177] gatherer "clusterconfig" function "version" took 35.471009ms to process 2 records I0706 01:08:27.195141 1 tasks_processing.go:74] worker 31 stopped. I0706 01:08:27.195322 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=0c35bb3921b0af355cd96b3dfce7d130632883a665f348eb34385cbb58db6c5e I0706 01:08:27.195404 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=a79fcbb5c07d0f3e5b2dd0bef109b044bbf10e16d95f713cedf1a4c11e264455 I0706 01:08:27.195413 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 36.273394ms to process 2 records I0706 01:08:27.196115 1 controller.go:212] Source scaController *sca.Controller is not ready I0706 01:08:27.196130 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0706 01:08:27.196134 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0706 01:08:27.196138 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0706 01:08:27.196142 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0706 01:08:27.196166 1 controller.go:489] The operator is still being initialized I0706 01:08:27.196177 1 controller.go:512] The operator is healthy I0706 01:08:27.198123 1 tasks_processing.go:74] worker 15 stopped. I0706 01:08:27.198188 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 43.839044ms to process 0 records I0706 01:08:27.199800 1 tasks_processing.go:74] worker 16 stopped. I0706 01:08:27.199866 1 recorder.go:75] Recording config/running_containers with fingerprint=2f3105ae5ccb8319c6b19db53aa789a8cc14f05b3b76305e9519cd2256890a5e I0706 01:08:27.199880 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 57.845077ms to process 1 records E0706 01:08:27.202028 1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27bd6b44fb-caaa-48f3-b262-e1724e429c62%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.11:35855->172.30.0.10:53: read: connection refused I0706 01:08:27.202041 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27bd6b44fb-caaa-48f3-b262-e1724e429c62%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.11:35855->172.30.0.10:53: read: connection refused I0706 01:08:27.204279 1 tasks_processing.go:74] worker 11 stopped. E0706 01:08:27.204291 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0706 01:08:27.204297 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0706 01:08:27.204302 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0706 01:08:27.204311 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0706 01:08:27.204340 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0706 01:08:27.204347 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0706 01:08:27.204351 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0706 01:08:27.204355 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0706 01:08:27.204392 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0706 01:08:27.204399 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0706 01:08:27.204404 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 76.829343ms to process 7 records I0706 01:08:27.251256 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload W0706 01:08:27.274749 1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.11:57632->172.30.0.10:53: read: connection refused I0706 01:08:27.274769 1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.130.0.11:57632->172.30.0.10:53: read: connection refused I0706 01:08:27.275839 1 base_controller.go:82] Caches are synced for LoggingSyncer I0706 01:08:27.275855 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0706 01:08:27.282960 1 tasks_processing.go:74] worker 25 stopped. E0706 01:08:27.282977 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0706 01:08:27.282984 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2rcar25f9ang6hc4cods4re8cimo7tgv-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2rcar25f9ang6hc4cods4re8cimo7tgv-primary-cert-bundle-secret" not found I0706 01:08:27.283086 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=8e101b8a335de99d185a3787ad2572aa558f87e22b5d9afc06169bff65a65ed4 I0706 01:08:27.283106 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 151.319571ms to process 1 records W0706 01:08:28.173446 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0706 01:08:28.387138 1 gather_cluster_operator_pods_and_events.go:121] Found 20 pods with 24 containers I0706 01:08:28.387235 1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1048576 bytes I0706 01:08:28.389686 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-924pq pod in namespace openshift-dns (previous: false). I0706 01:08:28.592571 1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found I0706 01:08:28.613061 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-924pq pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-924pq\" is waiting to start: ContainerCreating" I0706 01:08:28.613084 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-924pq\" is waiting to start: ContainerCreating" I0706 01:08:28.613095 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-924pq pod in namespace openshift-dns (previous: false). I0706 01:08:28.791076 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-924pq pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-924pq\" is waiting to start: ContainerCreating" I0706 01:08:28.791101 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-924pq\" is waiting to start: ContainerCreating" I0706 01:08:28.791117 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-gzchg pod in namespace openshift-dns (previous: false). I0706 01:08:29.016104 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-gzchg pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-gzchg\" is waiting to start: ContainerCreating" I0706 01:08:29.016125 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-gzchg\" is waiting to start: ContainerCreating" I0706 01:08:29.016135 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-gzchg pod in namespace openshift-dns (previous: false). W0706 01:08:29.173172 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0706 01:08:29.192145 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-gzchg pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-gzchg\" is waiting to start: ContainerCreating" I0706 01:08:29.192163 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-gzchg\" is waiting to start: ContainerCreating" I0706 01:08:29.192178 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-rngxm pod in namespace openshift-dns (previous: false). I0706 01:08:29.194646 1 tasks_processing.go:74] worker 19 stopped. I0706 01:08:29.194694 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=1f2dfefcbfd5926038416d1304c73f4d1d20f8d1ca8bef9b8e4b3475aee3a194 I0706 01:08:29.194728 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=e05d28be47bbcc6489fb64a387a0fe17c546f3b7dd37f7f9c8a342bf8f634c77 I0706 01:08:29.194756 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0706 01:08:29.194785 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=15d45764357a6a7110c04622b1631a208564bdcd6acdeccffebbb918c88fab27 I0706 01:08:29.194802 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0706 01:08:29.194824 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=eef83f8f8d42841b0b4c77c613a722265cc969745420516c4ef1f23e20133a91 I0706 01:08:29.194869 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=83e1657ea54bff86d2dc76d7d953ccfe3043d9f1d2fa558577d2cf88d7e6d49b I0706 01:08:29.194893 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=3c31df0ee49cba663a5576683f8922974dad034006ccb6d25ffaf25dcf657719 I0706 01:08:29.194907 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=43c0280b03851e8f6dbcc2c4bdea5500a0ea8701c1bfd661287fdd671eeaea6f I0706 01:08:29.194926 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=60e21055057263142897b3fae3cc453112db07123845800add8c4c64481f232b I0706 01:08:29.194936 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0706 01:08:29.194952 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=11ab4a29cbffcdd2b19af78545e615d878475d5c61386e6b986a78b47fa7881d I0706 01:08:29.194962 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0706 01:08:29.195001 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=ad0a6f9c87c1f6097fb2f53538dc7b09cae0b50d1121daf8d43afedf00fa1f2e I0706 01:08:29.195012 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0706 01:08:29.195034 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=63c132ccd6506636acb13d65fe9558eb4cbd6577606e573aea4c275c88ee5721 I0706 01:08:29.195044 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0706 01:08:29.195059 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=16b49f44f911e2624ef7b162d7d1244e7f08bd5a7e2d8ed06bf3a3f39f189b14 I0706 01:08:29.195179 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=b2654a92f9aef994c0870a990f0b9676fb30245712b6f2c34235fc65ce01f191 I0706 01:08:29.195188 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0706 01:08:29.195195 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0706 01:08:29.195214 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0706 01:08:29.195235 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=1aab4d4a25be6a7897f4dc4a967648e279c94eaf18dadaf9f095b572e04a2579 I0706 01:08:29.195258 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=132852f87ea33136ef2ded900747cda8e73b7b70562d35798ab12a51fee5c997 I0706 01:08:29.195268 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0706 01:08:29.195283 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=cbe877fd04d5243bc7687a0e23b9b0907cf9a3b81bcd293d94c396ce7be46dd9 I0706 01:08:29.195294 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0706 01:08:29.195307 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=d56fcd0e56fc885cbd35d00776689d93368e29af6624fbc167024ef7f3cb92bb I0706 01:08:29.195322 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=49ebc0739f2fbd1b9e8dc41a4082c2a84ae2b6316194f5d1c35e6e57c838646c I0706 01:08:29.195335 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=2f60bfd0be01879eb7529da15ea23e9d57caea2e08efe211ac093df626cd7e22 I0706 01:08:29.195350 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=e47eef1f3641388b5577a215ecef207e4b80f2345a4502ab9329e959959fc474 I0706 01:08:29.195373 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=161fccefce5745385aa2fc98687f9b5993ef6afbe09ed5f724b58852679cb119 I0706 01:08:29.195382 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0706 01:08:29.195409 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=1b3d9b95359656d116f2d3ae07a82377a872fce5e8722609a9daf98d84d1011e I0706 01:08:29.195425 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0706 01:08:29.195433 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0706 01:08:29.195439 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.065237329s to process 36 records I0706 01:08:29.409210 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-rngxm pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-rngxm\" is waiting to start: ContainerCreating" I0706 01:08:29.409228 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-rngxm\" is waiting to start: ContainerCreating" I0706 01:08:29.409236 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-rngxm pod in namespace openshift-dns (previous: false). I0706 01:08:29.591005 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-rngxm pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-rngxm\" is waiting to start: ContainerCreating" I0706 01:08:29.591028 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-rngxm\" is waiting to start: ContainerCreating" I0706 01:08:29.591038 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-2xrmm pod in namespace openshift-dns (previous: false). I0706 01:08:29.790427 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0706 01:08:29.790447 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-cfjsd pod in namespace openshift-dns (previous: false). I0706 01:08:29.992726 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0706 01:08:29.992746 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-jf9q6 pod in namespace openshift-dns (previous: false). W0706 01:08:30.174729 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0706 01:08:30.191629 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0706 01:08:30.191703 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-74b746c954-9wf2d pod in namespace openshift-image-registry (previous: false). I0706 01:08:30.390684 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-74b746c954-9wf2d pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-74b746c954-9wf2d\" is waiting to start: ContainerCreating" I0706 01:08:30.390708 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-74b746c954-9wf2d\" is waiting to start: ContainerCreating" I0706 01:08:30.390772 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-777ccf7cb9-j8qgd pod in namespace openshift-image-registry (previous: false). I0706 01:08:30.592323 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-777ccf7cb9-j8qgd pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-777ccf7cb9-j8qgd\" is waiting to start: ContainerCreating" I0706 01:08:30.592339 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-777ccf7cb9-j8qgd\" is waiting to start: ContainerCreating" I0706 01:08:30.592378 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-777ccf7cb9-nm446 pod in namespace openshift-image-registry (previous: false). I0706 01:08:30.791817 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-777ccf7cb9-nm446 pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-777ccf7cb9-nm446\" is waiting to start: ContainerCreating" I0706 01:08:30.791839 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-777ccf7cb9-nm446\" is waiting to start: ContainerCreating" I0706 01:08:30.791855 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-jwd5j pod in namespace openshift-image-registry (previous: false). I0706 01:08:30.993448 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0706 01:08:30.993472 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-kfb4n pod in namespace openshift-image-registry (previous: false). W0706 01:08:31.174765 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0706 01:08:31.190542 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0706 01:08:31.190564 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-lmz82 pod in namespace openshift-image-registry (previous: false). I0706 01:08:31.391177 1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty" I0706 01:08:31.391196 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-68bfc6bf6-9z77d pod in namespace openshift-ingress (previous: false). I0706 01:08:31.589910 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-68bfc6bf6-9z77d pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-68bfc6bf6-9z77d\" is waiting to start: ContainerCreating" I0706 01:08:31.589927 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-68bfc6bf6-9z77d\" is waiting to start: ContainerCreating" I0706 01:08:31.589938 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-68bfc6bf6-pg5dm pod in namespace openshift-ingress (previous: false). I0706 01:08:31.790861 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-68bfc6bf6-pg5dm pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-68bfc6bf6-pg5dm\" is waiting to start: ContainerCreating" I0706 01:08:31.790880 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-68bfc6bf6-pg5dm\" is waiting to start: ContainerCreating" I0706 01:08:31.790891 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-698c7c5547-2276q pod in namespace openshift-ingress (previous: false). I0706 01:08:31.990990 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-698c7c5547-2276q pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-698c7c5547-2276q\" is waiting to start: ContainerCreating" I0706 01:08:31.991008 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-698c7c5547-2276q\" is waiting to start: ContainerCreating" I0706 01:08:31.991021 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-fbhww pod in namespace openshift-ingress-canary (previous: false). W0706 01:08:32.173108 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0706 01:08:32.173141 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0706 01:08:32.173157 1 tasks_processing.go:74] worker 17 stopped. E0706 01:08:32.173168 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0706 01:08:32.173179 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 W0706 01:08:32.173193 1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics" I0706 01:08:32.173202 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.035895967s to process 1 records I0706 01:08:32.192240 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-fbhww pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fbhww\" is waiting to start: ContainerCreating" I0706 01:08:32.192258 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-fbhww\" is waiting to start: ContainerCreating" I0706 01:08:32.192268 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-mrg9s pod in namespace openshift-ingress-canary (previous: false). I0706 01:08:32.391382 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-mrg9s pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mrg9s\" is waiting to start: ContainerCreating" I0706 01:08:32.391400 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-mrg9s\" is waiting to start: ContainerCreating" I0706 01:08:32.391412 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-w7czv pod in namespace openshift-ingress-canary (previous: false). I0706 01:08:32.595061 1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-w7czv pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-w7czv\" is waiting to start: ContainerCreating" I0706 01:08:32.595089 1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-w7czv\" is waiting to start: ContainerCreating" I0706 01:08:32.595105 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for migrator container migrator-7d5f866c57-qqg5f pod in namespace openshift-kube-storage-version-migrator (previous: false). I0706 01:08:32.807221 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for graceful-termination container migrator-7d5f866c57-qqg5f pod in namespace openshift-kube-storage-version-migrator (previous: false). I0706 01:08:32.993560 1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-storage-version-migrator-operator container kube-storage-version-migrator-operator-74848b4cb9-wbwrv pod in namespace openshift-kube-storage-version-migrator-operator (previous: false). I0706 01:08:33.194159 1 tasks_processing.go:74] worker 5 stopped. I0706 01:08:33.194256 1 recorder.go:75] Recording events/openshift-dns-operator with fingerprint=c6ff2edcbca415697d7cdbb4a11d39b94b43c8f87edf6730c498cf48f35f04c3 I0706 01:08:33.194307 1 recorder.go:75] Recording events/openshift-dns with fingerprint=685814474131f8d20c9a9f51dc859151042802ea235ad4afbd5eb663da33e1d6 I0706 01:08:33.194409 1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=b4f9afecaa839e220a35133381b051c6a9fd3418ff142a1b691d08be736ac30d I0706 01:08:33.194436 1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=6320b832a4c8ff58d54efb823011f1435b363e748ef9e2afcb385304da645e62 I0706 01:08:33.194483 1 recorder.go:75] Recording events/openshift-ingress with fingerprint=b056c7233eeeb99977b13d270429876c64f41b1d6d8ec7c09cb21e40598f755e I0706 01:08:33.194507 1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=2eb7df4db8e8faa122e105d24f6dc790f7c6e432ae25d9d74113d4a38517ff68 I0706 01:08:33.194521 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator with fingerprint=037e03e7e4a7b6ee28fbfff0882f607cafd60784d02567689639a610928d2af5 I0706 01:08:33.194578 1 recorder.go:75] Recording events/openshift-kube-storage-version-migrator-operator with fingerprint=ecb4595a1a76492d8d097094fcc5ca755f4407ecd61221b60ec3b54f8a4d757d I0706 01:08:33.195928 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-74b746c954-9wf2d with fingerprint=3beb59bc412685fcc4623d6bed1ee686e68b179c1f52b0e0762ec60da471a3cb I0706 01:08:33.196072 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-777ccf7cb9-j8qgd with fingerprint=1735406672d2923fc54730f19fb6fe87e068043b0772b4aa666ec1f9adbdb620 I0706 01:08:33.196166 1 recorder.go:75] Recording config/pod/openshift-image-registry/image-registry-777ccf7cb9-nm446 with fingerprint=d8f9592092f711ab4692e92c526b1e1b88ec155c98b70f83a647d771dd013304 I0706 01:08:33.196182 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-qqg5f/migrator_current.log with fingerprint=7e51bd9a382a9022b26b4a83e59826257c71a2cacc3e4c90ff6e5c40f2b48338 I0706 01:08:33.196186 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator/logs/migrator-7d5f866c57-qqg5f/graceful-termination_current.log with fingerprint=7357a6729ae4382823e37842e0d43a258e8a8ee4e12091bb51f6e264c259e542 I0706 01:08:33.196259 1 recorder.go:75] Recording config/pod/openshift-kube-storage-version-migrator-operator/logs/kube-storage-version-migrator-operator-74848b4cb9-wbwrv/kube-storage-version-migrator-operator_current.log with fingerprint=207df94c25c9ac3fe05c7a9ab2176c4009a677fe8fd4752fcd4583f88c9ef2b0 I0706 01:08:33.196268 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 6.044634787s to process 14 records I0706 01:08:39.354500 1 configmapobserver.go:84] configmaps "insights-config" not found I0706 01:08:40.160516 1 tasks_processing.go:74] worker 14 stopped. I0706 01:08:40.160551 1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b I0706 01:08:40.160563 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.032780354s to process 1 records I0706 01:08:40.946394 1 tasks_processing.go:74] worker 27 stopped. I0706 01:08:40.946730 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=3a23a80037b37537e6bce19267a0686640cc3d2d84bcc6f7bc973979a947d17d I0706 01:08:40.946748 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.809254832s to process 1 records E0706 01:08:40.946837 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.819s with: function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"machines\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0706 01:08:40.947946 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "machine_healthchecks" failed with an error, function "machines" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0706 01:08:40.947960 1 periodic.go:209] Running workloads gatherer I0706 01:08:40.947975 1 tasks_processing.go:45] number of workers: 2 I0706 01:08:40.947983 1 tasks_processing.go:69] worker 1 listening for tasks. I0706 01:08:40.947988 1 tasks_processing.go:71] worker 1 working on workload_info task. I0706 01:08:40.947997 1 tasks_processing.go:69] worker 0 listening for tasks. I0706 01:08:40.948024 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0706 01:08:40.973454 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data I0706 01:08:40.981188 1 tasks_processing.go:74] worker 0 stopped. I0706 01:08:40.981201 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 33.141523ms to process 0 records I0706 01:08:40.982637 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (10ms) I0706 01:08:40.991519 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (9ms) I0706 01:08:41.000233 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (9ms) I0706 01:08:41.008637 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (8ms) I0706 01:08:41.016940 1 gather_workloads_info.go:387] No image sha256:e84cb128d930bd1ab867cc89b7b7bf2b2c0e41105ab93b5381069945b3ee9c57 (8ms) I0706 01:08:41.026205 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (9ms) I0706 01:08:41.035033 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (9ms) I0706 01:08:41.043546 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (9ms) I0706 01:08:41.052305 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (9ms) I0706 01:08:41.061349 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (9ms) I0706 01:08:41.083581 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (22ms) I0706 01:08:41.185702 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (102ms) I0706 01:08:41.283181 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (97ms) I0706 01:08:41.384020 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (101ms) I0706 01:08:41.483527 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (99ms) I0706 01:08:41.583234 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (100ms) I0706 01:08:41.683174 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (100ms) I0706 01:08:41.783383 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (100ms) I0706 01:08:41.862829 1 configmapobserver.go:84] configmaps "insights-config" not found I0706 01:08:41.882650 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (99ms) I0706 01:08:41.982859 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (100ms) I0706 01:08:42.062190 1 configmapobserver.go:84] configmaps "insights-config" not found I0706 01:08:42.082854 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (100ms) I0706 01:08:42.182833 1 gather_workloads_info.go:387] No image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (100ms) I0706 01:08:42.283103 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (100ms) I0706 01:08:42.382644 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (100ms) I0706 01:08:42.482913 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (100ms) I0706 01:08:42.583490 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0706 01:08:42.683844 1 gather_workloads_info.go:387] No image sha256:ca1344cb64140188b7cae7bbc51fb751566c0b0c97d5e39b5850e628032c4a5e (100ms) I0706 01:08:42.782617 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (99ms) I0706 01:08:42.884832 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (102ms) I0706 01:08:42.988300 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (103ms) I0706 01:08:43.082905 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (95ms) I0706 01:08:43.182888 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (100ms) I0706 01:08:43.182918 1 tasks_processing.go:74] worker 1 stopped. E0706 01:08:43.182928 1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset I0706 01:08:43.183234 1 recorder.go:75] Recording config/workload_info with fingerprint=56bf584e109f37333b36ef2fe227d3fb932c78565e4ba8df00a2211528b042ce I0706 01:08:43.183249 1 gather.go:177] gatherer "workloads" function "workload_info" took 2.234923076s to process 1 records E0706 01:08:43.183285 1 periodic.go:247] "Unhandled Error" err="workloads failed after 2.235s with: function \"workload_info\" failed with an error" I0706 01:08:43.184396 1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error I0706 01:08:43.184410 1 periodic.go:209] Running conditional gatherer I0706 01:08:43.193755 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0706 01:08:43.202437 1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.11:33388->172.30.0.10:53: read: connection refused E0706 01:08:43.202732 1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory I0706 01:08:43.202792 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0706 01:08:43.210962 1 conditional_gatherer.go:392] cluster version is '4.20.8' E0706 01:08:43.210976 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210981 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210985 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210988 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210991 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210994 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210997 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.210999 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing E0706 01:08:43.211002 1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing I0706 01:08:43.211016 1 tasks_processing.go:45] number of workers: 3 I0706 01:08:43.211033 1 tasks_processing.go:69] worker 2 listening for tasks. I0706 01:08:43.211039 1 tasks_processing.go:71] worker 2 working on remote_configuration task. I0706 01:08:43.211039 1 tasks_processing.go:69] worker 0 listening for tasks. I0706 01:08:43.211047 1 tasks_processing.go:69] worker 1 listening for tasks. I0706 01:08:43.211055 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0706 01:08:43.211056 1 tasks_processing.go:74] worker 1 stopped. I0706 01:08:43.211051 1 tasks_processing.go:71] worker 0 working on rapid_container_logs task. I0706 01:08:43.211094 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db I0706 01:08:43.211107 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 914ns to process 1 records I0706 01:08:43.211164 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406 I0706 01:08:43.211174 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 965ns to process 1 records I0706 01:08:43.211182 1 tasks_processing.go:74] worker 2 stopped. I0706 01:08:43.211318 1 tasks_processing.go:74] worker 0 stopped. I0706 01:08:43.211329 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 246.133µs to process 0 records I0706 01:08:43.211349 1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.130.0.11:33388->172.30.0.10:53: read: connection refused I0706 01:08:43.211366 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1 I0706 01:08:43.234240 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=8e8c82ecaffb791b3a3c8fb1ef6726da9bd2c7e7227b2dea2f81088d9b07f210 I0706 01:08:43.234364 1 diskrecorder.go:70] Writing 108 records to /var/lib/insights-operator/insights-2026-07-06-010843.tar.gz I0706 01:08:43.241327 1 diskrecorder.go:51] Wrote 108 records to disk in 7ms I0706 01:08:43.241355 1 periodic.go:278] Gathering cluster info every 2h0m0s I0706 01:08:43.241368 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0706 01:08:53.499442 1 configmapobserver.go:84] configmaps "insights-config" not found I0706 01:09:36.506199 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="3b15a737358a72615708a32bcf1a1fb3aed5d1508bdab4ef0780828d96cef2f4") W0706 01:09:36.506232 1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created I0706 01:09:36.506288 1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector I0706 01:09:36.506316 1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="315a258f95358c1a479796cdc44d09aaadaff036ad5b652813e42658b923015c") I0706 01:09:36.506370 1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated" I0706 01:09:36.506397 1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped" I0706 01:09:36.506418 1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="508c51f9b705a341308f39a98c20ba6e802cabdc2bade58182cdae533d7dff6b") I0706 01:09:36.506436 1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain" E0706 01:09:36.506454 1 controller.go:299] Unable to write cluster operator status: client rate limiter Wait returned an error: context canceled I0706 01:09:36.506459 1 periodic.go:170] Shutting down I0706 01:09:36.506460 1 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting" I0706 01:09:36.506467 1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0706 01:09:36.506471 1 requestheader_controller.go:194] Shutting down RequestHeaderAuthRequestController I0706 01:09:36.506486 1 base_controller.go:181] Shutting down ConfigController ... I0706 01:09:36.506491 1 tlsconfig.go:258] "Shutting down DynamicServingCertificateController"